Image capture device and vehicle

ABSTRACT

An image capture device is mounted in a vehicle. The image capture device includes: an image capture unit; and a setting unit that sets an image capture condition for each region of the image capture unit each having a plurality of pixels, or for each pixel, based upon at least one of a state exterior to the vehicle and a state of the vehicle.

TECHNICAL FIELD

The present invention relates to an image capture device and to avehicle.

BACKGROUND ART

A technique has been developed (refer to Patent Document #1) in whichthe traveling environment of a vehicle is detected on the basis ofimages acquired by a camera mounted upon the vehicle, and, on the basisof traveling environment data that has thus been detected, drivingsupport like automatic cruise control such as for following a leadingvehicle in front or the like, alarm provision, braking or steeringsupport, and so on is performed.

CITATION LIST Patent Literature

Patent Document #1: Japanese Laid-Open Patent Publication 2010-79424.

SUMMARY OF INVENTION Technical Problem

With prior art techniques, a solid-state imaging element such as a CCDor the like has been employed in the onboard camera. An onboard camerathat continually acquires images of the road or the like serves a veryimportant role in automatic cruise control or driving support or thelike, but there have not been many proposals for cameras that arespecifically intended to be mounted upon vehicles, and it has not beenpossible to say that the convenience of use of such cameras has beensufficient.

Although, with prior art techniques, lines upon the road have beendetected by using a camera, sometimes detection of such lines has becomedifficult in a traveling environment such as, for example, in a tunnelor during rainfall or the like.

From now on, it is predicted that automobiles that are performingautomatic travel control and automobiles that are traveling while beingmanually driven by drivers will be mixed together, but there have notbeen many proposals relating to this development.

Solution to Technical Problem

According to the 1st aspect of the present invention, an image capturedevice that is mounted to a vehicle, comprises: an image capture unit;and a setting unit that sets an image capture condition for each regionof the image capture unit each having a plurality of pixels, or for eachpixel, based upon at least one of a state exterior to the vehicle and astate of the vehicle.

According to the 2nd aspect of the present invention, in the imagecapture device according to the 1st aspect, the state of the vehicle maybe a traveling state of the vehicle.

According to the 3rd aspect of the present invention, in the imagecapture device according to 2nd aspect, the traveling state of thevehicle may be at least one of the direction of travel of the vehicle,and a speed of the vehicle.

According to the 4th aspect of the present invention, in the imagecapture device according to the 3rd aspect, it is preferable that atleast one of the direction of travel of the vehicle, and the speed ofthe vehicle, is controlled by the control unit of the vehicle.

According to the 5th aspect of the present invention, in the imagecapture device according to 3rd aspect, it is preferable that at leastone of the direction of travel of the vehicle, and the speed of thevehicle, is set by actuating an actuation unit of the vehicle.

According to the 6th aspect of the present invention, in the imagecapture device according to the 5th aspect, it is preferable that theactuation unit is at least one of a steering wheel, a turn signalswitch, an accelerator pedal, and a brake pedal.

According to the 7th aspect of the present invention, in the imagecapture device according to any one of the 4th through 6th aspects, itis preferable that at least one of the direction of travel of thevehicle, and the speed of the vehicle, is detected by a detection unitof the vehicle.

According to the 8th aspect of the present invention, in the imagecapture device according to any one of the 2nd through 7th aspects, itis preferable that the setting unit sets an imaging region to the imagecapture unit based upon the traveling state of the vehicle, and sets theimage capture condition for the imaging region that has been set.

According to the 9th aspect of the present invention, in the imagecapture device according to the 8th aspect, it is preferable that as theimage capture condition for the imaging region, the setting unit sets aframe rate to be higher than a frame rate of another region.

According to the 10th aspect of the present invention, in the imagecapture device according to the 8th or 9th aspect, it is preferable thatas the image capture condition for the imaging region, the setting unitsets a pixel decimation ratio to be lower than a decimation ratio foranother region.

According to the 11th aspect of the present invention, in the imagecapture device according to the 2nd aspect, it is preferable that thetraveling state of the vehicle is a distance from the vehicle to aleading vehicle ahead of the vehicle.

According to the 12th aspect of the present invention, in the imagecapture device according to the 11th aspect, it is preferable that thesetting unit sets an imaging region upon the image capture unit, andsets the image capture condition for the imaging region, based upon thedistance from the vehicle to the leading vehicle ahead of the vehicle.

According to the 13th aspect of the present invention, in the imagecapture device according to the 12th aspect, it is preferable that asthe image capture condition for the imaging region, the setting unitlowers a pixel decimation ratio to be lower than a decimation ratio foranother region.

According to the 14th aspect of the present invention, in the imagecapture device according to the 12th or 13th aspect, it is preferablethat as the image capture condition for the imaging region, the settingunit increases a frame rate to be higher than a frame rate for anotherregion.

According to the 15th aspect of the present invention, in the imagecapture device according to the 14th aspect, it is preferable that thesetting unit increases the frame rate for the imaging region if thedistance from the vehicle to the leading vehicle ahead of the vehiclebecomes shorter.

According to the 16th aspect of the present invention, in the imagecapture device according to the 12th or 15th aspect, it is preferablethat the setting unit increases a size of the imaging region if thedistance from the vehicle to the leading vehicle ahead of the vehiclebecomes shorter.

According to the 17th aspect of the present invention, in the imagecapture device according to the 1st aspect, it is preferable that thestate exterior to the vehicle is a state of a vehicle other than thevehicle.

According to the 18th aspect of the present invention, in the imagecapture device according to the 17th aspect, it is preferable that thestate of a vehicle other than the vehicle is a driving mode of a vehicleother than the vehicle.

According to the 19th aspect of the present invention, in the imagecapture device according to the 18th aspect, it is preferable that thedriving modes of the vehicle are an automatic driving mode in whichdriving is controlled by a control unit of the vehicle, and a manualdriving mode in which driving is performed by actuation of actuationunits of the vehicle.

According to the 20th aspect of the present invention, in the imagecapture device according to the 19th aspect, it is preferable that thesetting unit sets, upon the image capture unit, a region for capture ofan image of a vehicle operating in the manual driving mode and a regionfor capture of an image of a vehicle operating in the automatic drivingmode, and sets the image capture condition for the region that capturesimages of a vehicle operating in the manual driving mode and the imagecapture condition for the region that captures images of a vehicleoperating in the automatic driving mode to be different.

According to the 21st aspect of the present invention, in the imagecapture device according to the 20th aspect, it is preferable that asthe image capture condition, the setting unit raises a frame rate forthe region that captures the image of the vehicle operating in themanual driving mode to be higher than a frame rate for the region thatcaptures the image of the vehicle operating in the automatic drivingmode.

According to the 22nd aspect of the present invention, in the imagecapture device according to the 20th or 21st aspect, it is preferablethat as the image capture condition, the setting unit reduces adecimation ratio for the region that captures the image of the vehicleoperating in the manual driving mode to be lower than a decimation ratiofor the region that captures the image of the vehicle operating in theautomatic driving mode.

According to the 23rd aspect of the present invention, in the imagecapture device according to any one of the 18th or 22nd aspect, it ispreferable that the driving mode of the vehicle is detected by adetection unit.

According to the 24th aspect of the present invention, in the imagecapture device according to the 1st aspect, it is preferable that thestate exterior to the vehicle is a state of a road.

According to the 25th aspect of the present invention, in the imagecapture device according to the 24th aspect, it is preferable that thestate of the road is a state of a line that specifies a lane upon theroad along which the vehicle is traveling.

According to the 26th aspect of the present invention, in the imagecapture device according to the 25th aspect, it is preferable that theline is detected from an image captured by the image capture unit.

According to the 27th aspect of the present invention, in the imagecapture device according to the 25th or 26th aspect, it is preferablethat the setting unit sets a region upon the image capture unit forcapture of an image of the line, and sets the image capture conditionfor this region for capture of the image of the line.

According to the 28th aspect of the present invention, in the imagecapture device according to the 27th aspect, it is preferable that asthe image capture condition, the setting unit increases a frame rate forthe region that captures the image of the line to be higher than a framerate for a region other than the region that captures the image of theline.

According to the 29th aspect of the present invention, in the imagecapture device according to the 27th or 28th aspect, it is preferablethat as the image capture condition, the setting unit lowers a pixeldecimation ratio for the region that captures the image of the line tobe lower than a pixel decimation ratio for a region other than theregion that captures the image of the line.

According to the 30th aspect of the present invention, an image capturedevice that is mounted to a vehicle, comprises: an image capture unit;and a setting unit that sets a partial region within an imaging regionof the image capture unit corresponding to traveling of the vehicle, andan image capture condition for pixel of the partial region that has beenset, based upon at least one of a state exterior to the vehicle and astate of the vehicle.

According to the 31st aspect of the present invention, in the imagecapture device according to the 30th aspect, it is preferable that asthe image capture condition for the partial region that has been set,the setting unit raises a frame rate to be higher than a frame rate of aregion other than the partial region that has been set.

According to the 32nd aspect of the present invention, in the imagecapture device according to the 30th or 31st aspect, it is preferablethat as the image capture condition for the partial region that has beenset, the setting unit lowers a pixel decimation ratio to be lower than apixel decimation ratio of the region other than the partial region thathas been set.

According to the 33rd aspect of the present invention, in the imagecapture device according to the 30th or 32nd aspect, it is preferablethat the state of the vehicle is a traveling state of the vehicle.

According to the 34th aspect of the present invention, in the imagecapture device according to the 33rd aspect, it is preferable that thetraveling state of the vehicle is at least one of a direction of travelof the vehicle, and a speed of the vehicle.

According to the 35th aspect of the present invention, in the imagecapture device according to the 33rd aspect, it is preferable that thetraveling state of the vehicle is a distance from the vehicle to aleading vehicle ahead of the vehicle.

According to the 36th aspect of the present invention, in the imagecapture device according to the 30th aspect, it is preferable that thestate exterior to the vehicle is a state of a vehicle other than thevehicle.

According to the 37th aspect of the present invention, in the imagecapture device according to the 36th aspect, it is preferable that thestate of s vehicle other than the vehicle is a driving mode of a vehicleother than the vehicle.

According to the 38th aspect of the present invention, in the imagecapture device according to the 30th aspect, it is preferable that thestate exterior to the vehicle is a state of a road.

According to the 39th aspect of the present invention, in the imagecapture device according to the 38th aspect, it is preferable that thestate of the road is a state of a line that specifies a lane upon theroad along which the vehicle is traveling.

According to the 40th aspect of the present invention, a vehicle towhich an image capture device is mounted, comprises: a detection unitthat detects at least one of a state exterior to the vehicle and a stateof the vehicle; an image capture unit; arid a setting unit that sets apartial region within an imaging region of the image capture unitcorresponding to traveling of the vehicle, and an image capturecondition for pixel of the partial region that has been set, based uponat least one of a state exterior to the vehicle and a state of thevehicle, as detected by the detection unit.

According to the 41st aspect of the present invention, in the vehicleaccording to the 40th, it is preferable that the detection unit detectsa traveling state of the vehicle.

According to the 42nd aspect of the present invention, in the vehicleaccording to the 41st, it is preferable that as the traveling state ofthe vehicle, the detection unit detects at least one of a direction oftravel of the vehicle, and a speed of the vehicle.

According to the 43rd aspect of the present invention, in the vehicleaccording to the 41st, it is preferable that as the traveling state ofthe vehicle, the detection unit detects a distance from the vehicle to aleading vehicle ahead of the vehicle.

According to the 44th aspect of the present invention, in the vehicleaccording to the 43rd, it is preferable that as the traveling state ofthe vehicle, the detection unit detects the distance from the vehicle tothe leading vehicle ahead of the vehicle based upon an image captured bythe image capture unit.

According to the 45th aspect of the present invention, in the vehicleaccording to the 40th, it is preferable that the detection unit detectsa state of a vehicle other than the vehicle.

According to the 46th aspect of the present invention, in the vehicleaccording to the 45th, it is preferable that the detection unit detectsa driving mode of a vehicle other than the vehicle.

According to the 47th aspect of the present invention, in the vehicleaccording to the 40th, it is preferable that the detection unit detectsa state of a road.

According to the 48th aspect of the present invention, in the vehicleaccording to the 47th, it is preferable that the detection unit detectsthe state of the road based upon an image captured by the image captureunit.

According to the 49th aspect of the present invention, in the vehicleaccording to the 47th, it is preferable that as the stale of the road,the detection unit detects a state of a line that specifies a lane uponthe road along which the vehicle is traveling.

According to the 50th aspect of the present invention, in the vehicleaccording to any one of the 40th through 49th aspects, it is preferablethat as the image capture condition for the partial region that has beenset, the setting unit raises a frame rate to be higher than a frame ratefor a region other than the partial region that has been set.

According to the 51st aspect of the present invention, in the vehicleaccording to any one of the 40th through 50th aspects, it is preferablethat as the image capture condition for the partial region that has beenset, the setting unit lowers a pixel decimation ratio to be lower than apixel decimation ratio for a region other than the partial region thathas been set.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a figure showing the general structure of a driving supportdevice for a vehicle;

FIG. 2 is a block diagram showing an example of the structure of acontrol device;

FIG. 3 is a sectional view of a stacked imaging element;

FIG. 4 is a figure for explanation of a pixel array and unit regions onan image capture chip;

FIG. 5 is a figure for explanation of circuitry for a unit region;

FIG. 6 is a block diagram showing the functional structure of theimaging element;

FIG. 7 is a block diagram showing an example of the structure of acamera;

FIG. 8 is a figure showing in magnified view a region that includes aportion of a pixel line for focus detection;

FIG. 9 is a block diagram showing an example of the structure of acamera that includes an imaging element;

FIG. 10 is a figure showing examples of an imaging surface, an imagingregion and a region of attention, and an inactive region upon an imagecapture chip;

FIG. 11 is a flow chart for explanation of the flow of a camera controlprocedure executed by a control unit;

FIG. 12 is a flow chart for explanation of the details of initialsetting processing;

FIG. 13 is a figure showing an example of a table of initial settingvalues;

FIG. 14 is a figure showing examples of the imaging surface, the imagingregion and the region of attention, and the inactive region of the imagecapture chip;

FIG. 15 is a figure showing examples of the imaging surface, the imagingregion and the region of attention, and the inactive region of the imagecapture chip;

FIG. 16 is a figure showing examples of the imaging surface, the imagingregion and the region of attention, and the inactive region of the imagecapture chip;

FIG. 17 is a flow chart for explanation of the details of travelingassistance setting processing;

FIG. 18 is a figure for explanation of a flag Em;

FIG. 19 is a figure for explanation of distances Z;

FIG. 20(a) is a figure showing an example of shifting of the position ofthe region of attention and change of its size when a right turn is tobe made upon a normal road at an intersection, and FIG. 20(b) is afigure showing an example of shifting of the position of the region ofattention and change of its size when changing vehicle lane upon a highspeed road while accelerating;

FIG. 21 is a figure for explanation of a turn signal direction andchange of the size of the region for attention;

FIG. 22 is a flow chart for explanation of processing according to aVariant Embodiment #1 when a turn signal switch is actuated;

FIG. 23 is a figure schematically showing an image upon the imagingsurface of the image capture chip;

FIG. 24 is a flow chart for explanation of the overall flow of a cameracontrol procedure executed by the control unit;

FIG. 25 is a flow chart for explanation of the details of image capturecondition setting processing;

FIG. 26 is a flow chart showing an example of processing upon change toa first traveling environment;

FIG. 27 is a figure schematically showing an image upon the imagingsurface of the image capture chip;

FIG. 28 is a flow chart showing an example of processing upon change toa second traveling environment;

FIG. 29 consists of two figures schematically showing images upon theimaging surface of the image capture chip: FIG. 29(a) is a figureshowing a case of high beam, and FIG. 29(b) is a figure showing a caseof low beam;

FIG. 30 is a flow chart showing an example of processing upon change toa third traveling environment;

FIG. 31 is a figure schematically showing an image upon the imagingsurface of the image capture chip;

FIG. 32 is a flow chart showing an example of processing upon change toa fourth traveling environment;

FIG. 33 is a figure schematically showing an image upon the imagingsurface of the image capture chip;

FIG. 34 is a flow chart showing an example of processing upon change toa fifth traveling environment;

FIG. 35 consists of two figures schematically showing two images uponthe imaging surface of the image capture chip: FIG. 35(a) is a figureshowing a situation before change of vehicle lane, and FIG. 35(b) is afigure showing a situation during change of vehicle lane;

FIG. 36 is a block diagram showing the structure of an image capturesystem according to a third embodiment;

FIG. 37 is a figure showing an example of arrangement of traffic lightsat an intersection;

FIG. 38 is a figure showing an example of a traffic light forautomobiles;

FIG. 39 is a flow chart for explanation of control of an automobile by acontrol unit;

FIG. 40(a) is a figure for explanation of the positional relationship ofcertain automobiles, FIG. 40(b) is a figure schematically showing aphotographic subject image formed by a forward-facing camera, and FIG.40(c) is a figure schematically showing a photographic subject imageformed by a rearward-facing camera;

FIG. 41 is a flow chart for explanation of control of a traffic light bya control unit;

FIG. 42 is a figure for explanation of control of an image capture unitof a traffic light for automobiles;

FIG. 43 is a figure for explanation of control of an image capture unitof a traffic light for automobiles;

FIG. 44 is a figure for explanation of control of an image capture unitof a traffic light for pedestrians;

FIG. 45(a) is a figure showing an example of situation, an image ofwhich has been captured by an image capture unit installed to a trafficlight for pedestrians, and FIG. 45(b) is a figure for explanation ofsetting of image capture conditions; and

FIG. 46 is a figure showing an example of situation, an image of whichhas been captured by an image capture unit of a traffic light forautomobiles.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention will now be explained withreference to the drawings.

Embodiment #1 Situation in Use of a Camera

FIG. 1 is a figure showing the general structure of a driving supportdevice 2 of a vehicle 1 to which a camera 3 is mounted, according to afirst embodiment of the present invention. In FIG. 1, the drivingsupport device 2 is mounted to a vehicle 1 such as an automobile or thelike. The driving support device 2 comprises a camera 3, a controldevice 4, a first travel control unit 5, a second travel control unit 6,and so on.

It should be understood that although, in this explanation, an exampleis explained in which an internal combustion engine is taken as beingthe source of drive power, it would also be acceptable for an electricmotor to be the source of drive power, or the vehicle could be aso-called hybrid vehicle.

The camera 3 comprises an image capture optical system that has aplurality of lenses and an imaging element (in this embodiment, this isa stacked imaging element (refer to FIG. 3)), and that is attached, forexample, within the passenger compartment to the front of the roof. Thecamera 3 is pointed toward the front of the vehicle 1, and the height atwhich it is attached (i.e. the distance from the ground surface to thecamera 3) may, for example, be adjusted to 1.4 m. The camera 3 acquiresimages in the direction of travel of the vehicle 1, and, on the basis ofthe images that have thus been acquired, performs measurement (i.e.range finding) of the distances to various photographic subjects (i.e.objects) at a plurality of positions within the photographic screen.This distance measurement is calculated by range finding calculation,using the image signals from pixels for focus detection that areprovided upon the stacked imaging element. These pixels for focusdetection and range finding will be described hereinafter. The imagedata and range finding data acquired by the camera 3 are sent to thecontrol device 4. It should be understood that the camera 3 mayalternatively be provided external to the vehicle; or cameras 3 that areboth internal to and external to the vehicle may be used together incooperation; or an appropriate plural number of cameras may be provided.To cite examples, it will be acceptable to arrange for white linedetection that will be described hereinafter to be performed using acamera 3 that is external to the vehicle; and it will be acceptable toarrange for recognition of objects and/or obstructions to be performedin cooperation by cameras 3 that are both internal to and external tothe vehicle.

As shown in FIG. 2, the control device 4 includes a CPU 4 a and astorage unit 4 b. On the basis of programs of various types stored inthe storage unit 4 b, the CPU 4 a performs calculations of various typesusing control parameters stored in the storage unit 4 b and/or detectionsignals or the like from various sensors that will be describedhereinafter.

The first travel control unit 5 performs constant speed travelingcontrol and following travel control on the basis of commands from thecontrol device 4. Constant speed traveling control is control in whichthe vehicle 1 is made to travel at a constant speed on the basis of apredetermined control program. And following travel control is controlin which, while constant speed traveling control is being performed, ifthe speed of a leading vehicle in front that has been recognized by thecontrol device 4 is less than or equal to a target speed that has beenset for the vehicle 1, then the vehicle 1 is made to travel in a statein which it maintains a constant inter-vehicle distance with respect tothe leading vehicle.

And the second travel control unit 6 performs driving support control onthe basis of commands from the control device 4. This driving supportcontrol is control, on the basis of a predetermined control program, tooutput steering control signals to the steering control device 9 so asto make the vehicle 1 travel along the road, and control to output brakecontrol signals to the brake control device 8 so as to avoid collisionsbetween the vehicle 1 and various objects.

A throttle control device 7, a brake control device 8, a steeringcontrol device 9, a steering wheel 10, a turn signal switch 11, avehicle speed sensor 12, a yaw rate sensor 13, a display device 14, aGPS device 15, a shift lever position detection device 16, and amicrophone 17 are also shown in FIG. 1.

It should be understood that a beam change over switch 18 and a rainfallsensor 19 are structures that are not essential to this firstembodiment.

The throttle control device 7 controls the opening amount of a throttlevalve not shown in the figures, according to the amount by which anaccelerator pedal 7 a is stepped upon. Moreover the throttle controldevice 7 also performs, control of the opening amount of the throttlevalve mentioned above, according to a throttle control signal sent fromthe first travel control unit 5. The throttle control device 7 alsosends a signal specifying the amount by which the accelerator pedal 7 ais being stepped upon to the control device 4.

The brake control device 8 controls the opening amount of a brake valvenot shown in the figures according to the amount by which a brake pedal8 a is being stepped upon. Moreover, the brake control device 8 alsoperforms control of the opening amount of the brake valve mentionedabove, according to a brake control signal sent from the second travelcontrol unit 6. The brake control device 8 also sends a signalspecifying the amount by which the brake pedal 8 a is being stepped uponto the control device 4.

The steering control device 9 controls the steering angle of a steeringsystem not shown in the figures, according to the rotational angle ofthe steering wheel 10. Moreover, the steering control device 9 alsoperforms control of the steering angle of the steering system mentionedabove, according to a steering control signal sent from the secondtravel control unit 6. The steering control device 9 also sends signalsspecifying the rotational angle of the steering wheel 10 to the firsttravel control unit 5 and to the control device 4.

The turn signal switch 11 is a switch for operating turn signal devices(i.e. winkers) not shown in the figures. These turn signal devices areblinking light emission devices for indicating changes of the course ofthe vehicle 1. When the turn signal switch 11 is actuated by someone inthe vehicle 1, actuation signals from the turn signal switch 11 are sentto a turn signal device, to the second travel control unit 6, and to thecontrol device 4. And the vehicle speed sensor 12 detects the speed V ofthe vehicle 1, and sends its detection signal to the first travelcontrol unit 5, to the second travel control unit 6, and to the controldevice 4.

The yaw rate sensor 13 detects the rate of yawing of the vehicle 1, andsends its detection signal to the second travel control unit 6 and tothe control device 4. The rate of yawing is the rate of change of therotational angle of the vehicle 1 around its yaw direction. And thedisplay device 14 displays information showing the control states of thefirst travel control unit 5 and of the second travel control unit 6 andso on. This display device 14 may, for example, be built as a HUD (HeadUp Display) that projects information upon the windscreen. It should beunderstood that it would also be acceptable to utilize a display unit ofa navigation device not shown in the figures as the display device 14.

The UPS device 15 receives radio waves from the GPS satellites, andcalculates the position of the vehicle 1 (i.e. its latitude, longitude,and so on) by performing predetermined calculations using informationcarried by those radio waves. The position information calculated by theGPS device 15 is sent to a navigation device not shown in the figures,and to the control device 4. And the shift lever position detectiondevice 16 detects the position of a shift lever not shown in the figuresthat is actuated by someone riding in the vehicle 1 (for example to park(P), reverse (R), drive (D) and so on). Information specifying theposition of the shift lever detected by the shift lever positiondetection device 16 is sent to the control device 4.

The microphone 17 may, for example, include a front microphone, a rightside microphone, and a left side microphone. The front microphone hasdirectivity only to capture sound forward of the vehicle 1. The rightside microphone has directivity only to capture sound to the right sideof the vehicle 1. And the left side microphone has directivity only tocapture sound to the left side of the vehicle 1. The various streams ofaudio information captured by the microphone 17 (forward, to the rightside, and to the left side) are all sent to the control device 4.

Detection of Objects

The control device 4 performs image processing as described below uponthe images from the camera 3, in order to detect the road upon which thevehicle 1 is traveling and various objects. First, the control device 4creates a distance image (i.e. a depth distribution image) on the basisof the range finding data for a plurality of positions within thephotographic screen. And, on this basis of the distance image data, thecontrol device 4 performs per se known grouping processing, performscomparison with three dimensional frames (i.e. windows) of road shapedata, side wall data., object data and so on stored in advance in thestorage unit 4 b, and detects white line data (including data for whitelines extending along the road and data for white lines that cut acrossthe road (i.e. stop lines: intersection information)) and side wall datasuch as guard rails and curbs and so on present along the road, and alsodetects objects and obstructions, such as bicycles, ordinary vehicles,large sized vehicles, pedestrians, electricity poles, and so on, andclassifies them as objects of other types.

In the following explanation, both white colored and yellow coloredlines upon the path of travel will be termed “white lines”. Moreoversolid lines and broken lines will also be included as “white lines”.

Driving Support

On the basis of the information detected as described above, in otherwords the white line data, the guard rail side wall data, and the objectdata, the control device 4 recognizes objects and obstructions that areupon the path of travel or that may become obstacles, and performs thedriving support control described above for the second travel controlunit 6 on the basis of the results of this recognition. In other words,the control device causes the vehicle 1 to travel along the road, andcauses the vehicle 1 to avoid colliding with objects.

Travel Control

The control device 4 may, for example perform estimation of the path oftravel of the subject vehicle in the following four ways.

(1) Estimation of the Path of Travel of the Subject Vehicle on the Basisof White Lines

If white line data for both the left and right sides of the path oftraveling, or for only one of the left side and the right side thereof,is obtained from the images acquired by the camera 3, and if it ispossible to estimate the shape of the vehicle lane in which the vehicle1 is traveling from this white line data, then the control device 4estimates the path of travel of the subject vehicle in parallel with thewhite line or lines in consideration of the width of the vehicle 1 andthe position of the vehicle 1 within the current vehicle lane.

(2) Estimation of the Path of Travel of the Subject Vehicle on the Basisof Side Wall Data Such as a Guard Rail, a Curb or the Like

If side wall data for both the left side and the right side of the pathof traveling, or for only one of the left side and the right sidethereof, is obtained from the images acquired by the camera 3, and if itis possible to estimate the shape of the vehicle lane in which thevehicle 1 is traveling from this side wall data, then the control device4 estimates the path of travel of the subject vehicle in parallel withthe side wall or walls in consideration of the width of the vehicle 1and the position of the vehicle 1 within the current vehicle lane.

(3) Estimation of the Path of Travel of the Subject Vehicle on the Basisof the Track of a Leading Vehicle

The control device 4 estimates the path of travel of the subject vehicleon the basis of the past traveling track of a leading vehicle in front,which is stored in the storage unit 4 b. The leading vehicle is thatvehicle, among the objects that are traveling in the same direction asthe vehicle 1, to which the vehicle 1 is closest.

(4) Estimation of the Path of Travel of the Subject Vehicle on the Basisof the Traveling Track of the Vehicle 1

The control device 4 estimates the path of travel of the subject vehicleon the basis of the operational state of the vehicle 1. For example, onthe basis of the detection signal from the yaw rate sensor 13 and thedetection signal from the vehicle speed sensor 12, the path of travel ofthe subject vehicle may be estimated using the yaw curvature. The yawcurvature Cua is calculated according to the equation Cua=dψ/dt/V. Here,dψ/dt is the above described yaw rate (i.e. the rate of change of therotational angle in the yaw direction), and V is the speed of thevehicle 1.

According to a predetermined traveling control program stored in thestorage unit 4 b, for each of the objects described above, and on thebasis of the path of travel of the subject vehicle, the control device 4estimates the region of traveling of the vehicle 1 at the position wherethe object is present, compares this region of traveling with thepositions of the objects, and determines whether or not each of theobjects is within the region of traveling. Furthermore, the controldevice 4 recognizes the vehicle leading in front described above on thebasis of the result of image capture by the camera 3. In other words,from among the objects that are present within the region of travelingand that are traveling in the forward direction (i.e. the same directionas that of the vehicle 1), the control device 4 takes the closestvehicle to the vehicle 1 as being the leading vehicle.

The control device 4 outputs inter-vehicle distance information for theleading vehicle and the vehicle 1 and vehicle speed information for theleading vehicle to the first travel control unit 5 as informationrelating to conditions exterior to the vehicle. Here, the vehicle speedinformation for the leading vehicle is calculated on the basis of thevehicle speed V of the vehicle 1 which is acquired at predeterminedintervals, and change of the distance to the leading vehicle (i.e. ofthe inter-vehicle distance) which is range-found, at the predeterminedintervals described above, on the basis of images acquired by the camera3 in synchrony with the timings of acquisition of the vehicle speed V.

The first travel control unit 5 sends a throttle control signal to thethrottle control device 7, so that the vehicle speed V detected by thevehicle speed sensor 12 converges to a predetermined vehicle speed (i.e.to a target speed) that is set in advance. Due to this, the throttlecontrol device 7 feedback controls the opening amount of the throttlevalve not shown in the figures, so as to make the vehicle 1 travelautomatically at a constant speed.

Moreover if, while traveling control is being performed in the constantspeed state, the vehicle speed information for the leading vehicleinputted from the control device 4 is less than or equal to the targetspeed set for the vehicle 1, then the first travel control unit 5outputs a throttle control signal to the throttle control device 7 onthe basis of the inter-vehicle distance information inputted from thecontrol device 4. In concrete terms, the first travel control unit 5sets an appropriate target value for the inter-vehicle distance on thebasis of the inter-vehicle distance from the vehicle 1 to the leadingvehicle, the vehicle speed of the leading vehicle, and the vehicle speedV of the vehicle 1, and sends a throttle control signal to the throttlecontrol device 7, so that the inter-vehicle distance that is range-foundon the basis of the images acquired by the camera 3 converges to thetarget value for the inter-vehicle distance described above. Due tothis, the throttle control device 7 feedback controls the opening amountof the throttle valve not shown in the figures, so as to cause thevehicle 1 to travel while following the leading vehicle in front.

Explanation of a Stacked Imaging Element

A stacked imaging element 100 incorporated in the camera 3 describedabove will now be explained. It should be understood that this stackedimaging element 100 is of a type described in International PublicationWO13/164915, previously filed by the same applicant as the presentapplication. FIG. 3 is a sectional view of this stacked imaging element100. The imaging element 100 comprises a backside-illumination typeimage capture chip 113 that outputs pixel signals corresponding toincident light, a signal processing chip 111 that processes these pixelsignals, and a memory chip 112 that stores the pixel signals. This imagecapture chip 113, signal processing chip 111, and memory chip 112 arelaminated together, and are mutually electrically connected together bybumps 109 that are made from Cu or the like and that are electricallyconductive.

It should be understood that, as shown in the figure, incident light isprincipally incident along the +Z axis direction shown by the outlinedwhite arrow sign. In this embodiment, the surface of the image capturechip 113 on its side upon which the incident light is incident (i.e. itsimaging surface) will be referred to as its rear surface. Furthermore,as shown by the coordinate axes in the figure, the direction leftwardupon the drawing paper and orthogonal to the Z axis will be taken asbeing the +X axis direction, and the direction toward the viewer fromthe drawing paper and orthogonal to the Z axis and to the X axis will betaken as being the +Y axis direction. In some of the following figures,coordinate axes are shown so that the orientations of these figures canbe understood with respect to the reference coordinate axes of FIG. 3.

One example of the image capture chip 113 is an MOS image sensor of thebackside-illumination type A PD layer is disposed on the rear surfaceside of a wiring layer 108. This PD layer 106 comprises a plurality ofPDs (photodiodes) that are arranged two-dimensionally and thataccumulate charge according to incident light, and transistors 105provided to correspond to these PDs 104.

Via a passivation layer 103, color filters 102 are provided upon theincident light side of the PD layer 106. These color filters 102 are ofa plurality of types that pass mutually different wavelength regions,and they respectively correspond to the PDs 104 and are arrayed in aspecific arrangement. The arrangement of the color filters 102 will bedescribed hereinafter. Each group consisting of a color filter 102, a PD104, and a transistor 105 constitutes a single pixel.

A micro lens 101 is provided corresponding to each of these pixels, onthe incident light side of its color filter 102. This micro lens 101condenses incident light upon its corresponding PD 104.

A wiring layer 108 includes wiring 107 that transmits the pixel signalsfrom the PD layer 106 to the signal processing chip 111. This wiring 107may be multi-layered, and could also include both passive elements andactive elements.

A plurality of bumps 109 are disposed on the surface of the wiring layer108. The positions of this plurality of bumps 109 are matched to thepositions of a plurality of other bumps 109 that are provided upon theopposing surface of the signal processing chip 111, and are joined tothe bumps 109 with which they are positionally aligned and areelectrically connected thereto by the image capture chip 113 and thesignal processing chip 111 being put under pressure, or the like.

In a similar manner, a plurality of bumps 109 are disposed upon themutually opposing surfaces of the signal processing chip 111 and thememory chip 112. Due to these bumps 109 being mutually positionallyaligned, and due to pressure being applied to the signal processing chip111 and the memory chip 112 or the like, these bumps 109 that aremutually positionally aligned with one another are joined together andare electrically connected together.

It should be understood that the joining between the bumps 109 is notlimited to Cu bump joining due to solid phase diffusion; it would alsobe acceptable to connect the micro bumps together by solder melting.Moreover, for example, it will be sufficient to provide around one bump109 or so for each one block that will be described hereinafter.Accordingly the size of the bumps 1 09 may be greater than the pitch ofthe PDs 104. Also, bumps that are larger than the bumps 109corresponding to the pixel regions may be provided together inperipheral regions other than the pixel regions in which the pixels arearranged.

The signal processing chip 111 is provided with TSVs (Trans SiliconVias) 110 that mutually connect circuits provided upon its front surfaceand upon its rear surface. It is desirable for the TSVs to be providedin peripheral regions. Moreover, TSVs 110 are also provided inperipheral regions of the image capture chip 113 and the memory chip112.

FIG. 4 is a figure for explanation of the pixel array and the unitregions 131 of the image capture chip 113. In particular, this figureshows the situation when this image capture chip 113 is observed fromits rear surface (i.e. from its imaging surface). For example,20,000,000 or more pixels may be arranged in this pixel region in theform of a matrix. In the FIG. 4 example, 4×4 adjacent pixels, i.e. 16adjacent pixels, constitute one unit region 131. The lattice lines inthe figure conceptually show the way in which each unit region 131 isformed by grouping adjacent pixels together. The number of pixels thatform each unit region 131 is not limited to being as above; for example,there could be around 1000 pixels arranged as 32×64 pixels, or therecould be more or fewer thereof.

As shown in the enlarged view of a part of the pixel region, a number ofso-called Bayer arrays are included in the unit region 131 in FIG. 4,arranged both vertically and horizontally, and each being made up fromfour pixels two green colored pixels Gb and Gr, a blue colored pixel B,and a red colored pixel R. The green colored pixels Gb and Gr are pixelsthat have green colored filters as their color filters 102, and thatreceive light in the green colored wavelength band from the incidentlight. In a similar manner, the blue colored pixels B are pixels thathave blue colored filters as their color filters 102 and that receivelight in the blue colored wavelength band, while the red colored pixelsR are pixels that have red colored filters as their color filters 102and that receive light in the red colored wavelength band.

In this embodiment, a plurality of blocks are defined so that at leastone unit region 131 is included in each block, and each of these blockscan control the pixels included in it with control parameters that aremutually different. In other words, with the pixel groups belonging tosome block and the pixel groups belonging to a different block, it ispossible to acquire image capture signals fir which the image captureconditions are different. Examples of such control parameters are framerate, gain, decimation ratio, number of rows or number columns foraddition of the pixel signals, charge accumulation time or number oftimes for accumulation, number of digitized bits (i.e. word length), andso on. This imaging element 100 not only performs decimation in the rowdirection (the X axis direction on the image capture chip 113), but alsocan, at will, perform decimation in the column direction (the Y axisdirection on the image capture chip 113). Furthermore, a controlparameter could also be a parameter for image processing after the imagesignals from the pixels have been acquired.

FIG. 5 is a figure for explanation of the circuit for one of the unitregions 131. In the FIG. 5 example, one unit region 131 is constitutedby 3×3 adjacent pixels, i.e. by 9 adjacent pixels. It should beunderstood that the number of pixels that are included in each unitregion 131 as described above is not limited to being nine as above;there could be fewer or more. The two dimensional positions of thepixels in the unit region 131 are denoted by the reference symbols Athrough I.

Reset transistors for the pixels included in the unit region 131 areprovided so that they can be turned ON and OFF individually. In FIG. 5,a reset wire 300 is provided for turning the reset transistor for thepixel A ON and OFF, and a reset wire 310 for turning the resettransistor for the pixel B ON and. OFF is provided separately from theabovementioned reset wire 300. In a similar manner, a reset wire 320 forturning the reset transistor for the pixel C ON and OFF is providedseparately from the abovementioned reset wires 300 and 310. Dedicatedreset wires are also similarly provided for turning the respective resettransistors for the other pixels D through I ON and OFF.

Transfer transistors for the pixels included in the unit region 131 arealso provided for each of the pixels individually, so that they can beturned ON and OFF. In FIG. 5, a transfer wire 302 for turning thetransfer transistor for the pixel A ON and OFF, a transfer wire 312 forturning the transfer transistor for the pixel B ON and OFF, and atransfer wire 322 for turning the transfer transistor for the pixel C ONand OFF are provided separately. Dedicated transfer wires are alsoprovided for turning the respective transfer transistors for the otherpixels D through I ON and OFF.

Furthermore, selection transistors for the pixels included in the unitregion 131 are also provided so that they can be turned ON and OFF foreach of the pixels individually. In FIG. 5, a selection wire 306 forturning the selection transistor for the pixel A ON and OFF, a selectionwire 316 for turning the selection transistor for the pixel B ON andOFF, and a selection wire 326 for turning the selection transistor forthe pixel C ON and OFF are provided separately. Dedicated selectionwires are also provided for turning the respective selection transistorsfor the other pixels D through I ON and OFF.

It should be understood that a power supply wire 304 is connected to allthe pixels A through I included in the unit region 131 in common. In asimilar manner, an output wire 308 is connected to all the pixels fromthe pixel A through I included in the unit region 131 in common.Moreover, while the power supply wire 304 is connected between aplurality of unit regions in common, an individual output wire 308 isprovided for each of the output regions 131. A load current source 309May be provided on the side of the image capture chip 113, or may beprovided on the side of the signal processing chip 111.

By turning the reset transistors and the transfer transistors of theunit region 131 ON and OFF individually, it is possible to controlcharge accumulation for the pixels A through I included in the unitregion 131 independently, including their times of starting chargeaccumulation, their times of ending charge accumulation, and theirtransfer timings. Furthermore, by turning the selection transistors ofthe unit region 131 ON and OFF individually, it is possible to outputpixel signals for the pixels A through I via the common output wire 308.

Here, the so-called rolling shutter method is per se known forcontrolling charge accumulation for the pixels A through I included inthe unit region 131 in a regular sequence for rows and columns. When,according to the rolling, shutter method, the columns are specifiedafter having selected each row of pixels, then, in the example shown inFIG. 5, the pixel signals are outputted in the order “ABCDEFGHI”.

By taking the unit regions 131 as reference and building the circuit inthis manner, it is possible to control the charge accumulation time foreach unit region 131. To put this in another manner, it is possible tooutput pixel signals according to different frame rates for each of theunit regions 131. Moreover, due to the fact that, while chargeaccumulation (image capture) is being performed by a unit region 131that is included in one partial area upon the image capture chip 113, aunit region included in another area is being rested, it is possible toperform image capture with only a predetermined area of the imagecapture chip 113, and to output pixel signals for that area.Furthermore, by changing over the area for which charge accumulation(image capture) is performed (i.e. the subject area for accumulationcontrol) between frames, it is possible to output pixel signals byperforming image capture sequentially for different areas of the imagecapture chip.

FIG. 6 is a block diagram showing the functional structure of an imagingelement 100 corresponding to the circuit shown in the FIG. 5 example. AnAnalog multiplexer 411 selects the 9 PDs 104 that make up the unitregion 131 in order, and outputs their respective pixel signals to theoutput wire 308 that is provided to correspond to that unit region 131.The multiplexer 411 is provided upon the image capture chip 113 togetherwith the PDs 104.

The pixel signals outputted via the multiplexer 411 are CDS and A/Dconverted by a signal processing circuit 412 that is formed upon thesignal processing chip 111 and that performs correlated double sampling(CDS) and analog/digital (A/D) conversion. After having been A/Dconverted, the pixel signals are passed to a demultiplexer 413, and arestored in pixel memories 414 corresponding to each of the pixels. Thedemultiplexer 413 and the pixel memories 414 are formed upon the memorychip 112.

A calculation circuit 415 processes the pixel signals stored in thepixel memories 414 and passes them over to an image processing unitwhich is a subsequent stage. This calculation circuit 415 could beprovided upon the signal processing chip 111, or could be provided uponthe memory chip 112.

It should be understood that, while in FIG. 6 only the connections for asingle unit region 131 are shown, actually such connections are providedfor each unit region 131, and operate in parallel. However, it wouldalso be acceptable not to provide a calculation circuit 415 for eachunit region 131, but, for example, for a single calculation circuit 415to perform sequential processing while referring in order to the valuesin the pixel memories 414 corresponding to each unit region 131.

As described above, an output wire 308 is provided to correspond to eachof the unit regions 131. Since, in this imaging element 100, the imagecapture chip 113, the signal processing chip 111, and the memory chip112 are laminated together, accordingly it is possible to route thewiring without enlarging the chips in the surface direction by employingelectrical connections between these chips using the bumps 109.

Explanation of the Range Finding

FIG. 7 is a figure showing an example of the positions of pixels forfocus detection upon the imaging surface of the imaging element 100. Inthis embodiment, pixels for focus detection are provided in separatelines along the X axis direction of the image capture chip 113 (i.e.along its horizontal direction). In the FIG. 7 example, fifteen focusdetection pixel lines are provided at predetermined intervals. Thepixels for focus detection that make up the focus detection pixel lines60 output image signals for range finding. Normal pixels for imagecapture are provided at the pixel positions on the image capture chip113 other than these focus detection pixel lines 60. These pixels forimage capture perform monitoring for moving objects or obstructionsexternal to the vehicle and output image signals for extra-vehiclemonitoring.

FIG. 8 is a figure showing in magnified view a region that includes aportion of one of the focus detection pixel lines 60 described above. InFIG. 8, red colored pixels R, green colored pixels G (Gb and Gr), bluecolored pixels B, pixels for focus detection S1, and pixels for focusdetection 52 are shown by way of example. The red colored pixels R, thegreen colored pixels G (Gb and Gr), and the blue colored pixels B arearranged according to the rule for a Bayer array described above.

The square shaped regions that are shown by way of example for the redcolored pixels R, the green colored pixels G (Gb and Gr), and the bluecolored pixels B are light reception regions of the pixels for imagecapture. The pixels for image capture receive light fluxes through theexit pupil of an image capture optical system 31 (refer to FIG. 9). Inother words, each of the red colored pixels R, each of the green coloredpixels G (Gb and Gr), and each of the blue colored pixels B has a squareshaped mask opening portion, and light that passes these mask openingportions arrives at the respective light reception sections of thepixels for image capture.

It should be understood that the shapes of the light reception regions(i.e. of the mask opening portions) of the red colored pixels R, thegreen colored pixels G (Gb and Gr), and the blue colored pixels B arenot limited to being quadrilateral; they could, for example, becircular.

Semicircular shaped regions on the pixels for focus detection S1 and onthe pixels for focus detection S2 indicate the light receiving regionsof these pixels for focus detection. In other words, the pixels forfocus detection S1 have semicircular shaped mask opening portions on theleft sides of their pixel positions in FIG. 8, and light that has passedthrough these mask opening portions reaches the light reception sectionsof the pixels for focus detection S1. On the other hand, the pixels forfocus detection S2 have semicircular shaped mask opening portions on theright sides of their pixel positions in FIG. 8, and light that haspassed through these mask opening portions reaches the light receptionsections of the pixels for focus detection S2. In this manner, each ofthe pixels for focus detection S1 and the pixels for focus detection S2receives one of a pair of light fluxes that have passed throughdifferent regions of the exit pupil of the image capture optical system31 (refer to FIG. 9).

It should be understood that the positions of the focus detection pixellines upon the image capture chip 113 are not to be considered as beinglimited to the positions shown by way of example in FIG. 7. Moreover,the number of the focus detection pixel lines is also not to beconsidered as being limited by the FIG. 7 example. Yet further, theshapes of the mask opening portions of the pixels for focus detection S1and of the pixels for focus detection S2 are not to be considered asbeing limited to being semicircular; for example, it would also beacceptable to arrange to form them in rectangular shapes which areobtained by dividing the quadrilateral shaped light receiving regions(i.e. the mask opening portions) on certain ones of the pixels R forimage capture, the pixels G for image capture, or the pixels B for imagecapture in the horizontal direction.

Furthermore, the focus detection pixel lines on the image capture chip113 could also be provided as lined up along the Y axis direction (i.e.the vertical direction) of the image capture chip 113. An imagingelement upon which pixels for image capture and pixels for focusdetection are arrayed two dimensionally as in FIG. 8 is per se known,and accordingly the details of these pixels are not shown in the figureand will not be explained.

It should be understood that, in the FIG. 8 example, a so-called 1PDconstruction is explained in which each of the pixels for focusdetection S1 and S2 receives one of a pair of light fluxes for focusdetection. Instead of this as for example disclosed in JapaneseLaid-Open Patent Publication 2007-282107, it would also be acceptable toarrange to employ a so-called 2PD construction, in which each of thepixels for focus detection receives both of a pair of light fluxes forfocus detection. By using a 2PD construction in this manner, it becomespossible to read out image data from the pixels for focus detection aswell, so that the focus detection pixels do not become defective pixels.

In this embodiment, on the basis of the image signals for range findingthat are outputted from the pixels for focus detection S1 and the pixelsfor focus detection S2, the focus adjustment state (i.e. the defocusingamount) of the image capture optical system 31 (refer to FIG. 9) iscalculated by detecting the amount of image deviation (i.e. the phasedifference) between a pair of images due to a pair of light fluxes thathave passed through different regions of the image capture opticalsystem.

Generally, in a so-called front focused state in which the image captureoptical system 31 focuses a sharp image of an object (for example, of aleading vehicle in front) ahead of a prearranged focal plane, the pairof images described above are closer to one another; and, conversely, ina so-called back focused state in which a sharp image of the object isfocused behind the prearranged focal plane, they are further away fromone another. And in a focused state in which a sharp image of the objectis at the prearranged focal plane, the pair of images described aboverelatively coincide with one another. Accordingly the relative amount ofpositional deviation between the pair of objects corresponds to thedistance (depth information) to the object.

Since calculation of the defocusing amount on the basis of the phasedifference described above is per se known in the camera field,accordingly detailed explanation thereof will be omitted. Here, it ispossible to obtain the distances from the camera 3 to various objects byobtaining the amount of defocusing for each object, since the defocusingamount and the distance to the object are in one-to-one correspondence.In other words, distance measurement (i.e. range finding) is performedto each of the objects mentioned above at a plurality of positions inthe photographic screen. The relationship between the amount ofdefocusing and the distance to the object is prepared in advance as anequation or as a lookup table, and is stored in a non-volatile memory 35b (refer to FIG. 9).

Explanation of the Camera

FIG. 9 is a block diagram showing an example of the structure of acamera 3 that incorporates the imaging element 100 described above. InFIG. 9, the camera 3 comprises an image capture optical system 31, animage capture unit 32, an image processing unit 33, a working memory 34,a control unit 35, and a recording unit 36.

The image capture optical system 31 conducts a light flux from thephotographic field to the image capture unit 32. This image capture unit32 includes the imaging element 100 described above and a drive unit 32a, and photoelectrically converts the image of an object that has beenfocused upon the image capture chip 113 by the image capture opticalsystem 31. The drive unit 32 a generates the necessary drive signals forperforming charge accumulation control independently for each of theblock units described above upon the imaging element 100 (i.e. upon theimage capture chip 113). Commands for the positions and the shapes ofthe blocks described above, for their ranges, for heir chargeaccumulation times, and so on are transmitted from the control unit 35to the drive unit 32 a.

In cooperation with the working memory 34, the image processing unit 33performs image processing upon the image data captured by the imagecapture unit 32. For example, in addition to performing image processingsuch as contour enhancement processing and gamma correction and so on,the image processing unit 33 may also perform color detection forobjects included in the image.

The working memory 34 temporarily stores the image data and so on beforeand after processing. And the recording unit 36 records the image dataand so on upon a storage medium that consists of a non-volatile memoryor the like. The control unit 35 may, for example, be built around aCPU, and controls the entire operation of the camera 3 according tocontrol signals from the control device 4. For example, the control unitmay perform predetermined exposure calculation on the basis of the imagesignals captured by the image capture unit 32, and may command the driveunit 32 a to provide times for charge accumulation by the image capturechip 113 as required for appropriate exposure.

The control unit 35 includes a range finding calculation unit 35 a andthe non-volatile memory 35 b. As described above, the range findingcalculation unit 35 a performs measurement of the distance to (i.e.finds the range of) each of the objects described above at a pluralityof positions in the photographic screen. And the image data that hasbeen acquired by the camera 3 and the range finding data that has beencalculated by the camera 3 are sent to the control device 4 (refer toFIG. 1). The non-volatile memory 35 b stores the program executed by thecontrol unit 35 a and the information required for range finding.

Control of the Imaging Element Blocks

The control device 4 causes the imaging element 100 (i.e. the imagecapture chip 113) of the camera 3 to perform charge accumulation controlindependently for each of the block units described above. For doingthis, the following signals are inputted from the various sections ofthe vehicle 1 to the control device 4 (refer to FIG. 2).

(1) The Amount by Which the Accelerator Pedal 7 a is Being Stepped Upon

A signal that specifies the amount by which the accelerator pedal 7 a isbeing stepped upon is inputted from the throttle control device 7 to thecontrol device 4.

(2) The Amount by Which the Brake Pedal 8 a is Being Stepped Upon

A signal that specifies the amount by which the brake pedal 8 a is beingstepped upon is inputted from the brake control device 8 to the controldevice 4.

(3) The Rotational Angle of the Steering Wheel 10

A signal that specifies the rotational angle of the steering wheel 10 isinputted from the steering control device 9 to the control device 4. Theratio of the rotational angle of the steering wheel 10 to the steeringangle of the steering system depends upon the gearing ratio of thesteering.

(4) The Speed V of the Vehicle 1

The detection signal from the vehicle speed sensor 12 is inputted to thecontrol device 4.

(5) The Actuation Signal of the Turn Signal Switch 11

The actuation signal of the turn signal switch 11 is inputted to thecontrol device 4.

(6) The Position to Which the Shift Lever is Operated

A signal is inputted to the control device 4 specifying the position towhich the shift lever is actuated, as detected by the shift leverposition detection device 16.

(7) The Position Information for the Vehicle 1

Position information that has been measured by the UPS device 15 isinputted from the GPS device 15 to the control device 4.

(8) Information About Sound Around the Vehicle 1

Sound information captured by the microphone 17 from the front of thevehicle, from its right side, and from its left side is inputted to thecontrol device 4.

FIG. 10 is a figure showing examples of the imaging surface of the imagecapture chip 113, regions (an imaging region 81 and a region ofattention 82) upon the image capture chip 113 in which chargeaccumulation (i.e. image capture) is performed, and a region (aninactive region 83) in which charge accumulation (i.e. image capture) inthe row direction and the column direction is not performed. The regionof attention 82 is a region in which charge accumulation (i.e. imagecapture) is performed under different conditions from those in theimaging region 81. The sizes and positions of the imaging region 81 andthe region of attention 82 upon the image capture chip 113 are alsoincluded in the image capture conditions.

The control device 4 executes control by setting first conditions foreach of the unit regions 131 included in the imaging region 81 so thatthey perform image capture, and also executes control by setting secondconditions for each of the unit regions 131 included in the region ofattention 82 so that they perform image capture. Moreover, the controldevice does not execute any activity in connection with the unit regions131 included in the inactive region 83, so that they do not performimage capture.

It should be understood that it would also be acceptable to provide aplurality of regions of interest 82, and it would be acceptable for theconditions for image capture to be different between each of thisplurality of regions of interest. Moreover, it would also be acceptablenot to provide any inactive region 83.

Explanation of the Flow Charts

Now, the way in which the imaging region 81 and the region of attention82 are determined will be explained in the following with reference tothe flow charts (FIGS. 11, 12, and 17). FIG. 11 is a flow chart forexplanation of the flow of a control procedure for the camera 3 that isexecuted by the control device 4. The program for executing theprocessing of the flow chart of FIG. 11 is stored in the storage unit 4b of the control device 4. The control device 4 may, for example, startthe program for performing the processing of FIG. 11 when the supply ofpower from the vehicle 1 is started, or when the engine is started.

In step S10 of FIG. 11, the control device 4 makes a decision as towhether or not a flag a=0. The flag a is a flag that is set to 1 ifinitial setting has terminated, and that is set to 0 if initial settinghas not yet terminated. If the flag a=0 so that the control device 4reaches an affirmative decision in step S10, then the flow of controlproceeds to step S20, whereas if the flag a≠0 so that a negativedecision is reached in the step S10 then the flow of control istransferred to a step S30.

In step S20 the control device 4 performs initial setting processing,and then the flow of control proceeds to step S30. The details of thisinitial setting processing will be described hereinafter. In step S30,the control device 4 performs traveling assistance setting processing,and then the flow of control proceeds to step S40. In the travelingassistance setting processing, an imaging region 81 and a region ofattention 82 upon the imaging element 100 are determined. The details ofthis traveling assistance setting processing will be describedhereinafter.

In step S40, the control device 4 sends a command to the camera 3, anddrives the imaging region 81 and the region of attention 82 on theimaging element 100 under respective predetermined conditions, so as toperform acquisition of an image. In this embodiment, for example, as thevehicle speed V increases from zero, the control device 4 may set theframe rate of the region of attention 82 to be higher, its gain to behigher, its decimation ratio to be lower, and its charge accumulationtime to be shorter, as compared to the case for the imaging region 81.Along with image capture being performed by the camera 3 in this manner,also distance measurement (i.e. range finding) is performed at aplurality of positions in the photographic screen, as described above.

It should be understood that it is not necessary for all of the framerate, the gain, the decimation ratio, the charge accumulation time andso on to be different between the imaging region 81 and the region ofattention 82; it will be acceptable if only at least one of them isdifferent. Moreover, it would also be acceptable for the control device4 to establish a setting for no decimation to be performed for theregion of attention 82.

In step S45, the control device 4 acquires the image data and the rangefinding data from the camera 3, and then the flow of control proceeds tostep S50. In step S50, the control device 4 makes a decision as towhether or not a setting for information display is established. Ifdisplay setting is established, then the control device 4 reaches anaffirmative decision in step S50, and the flow of control proceeds tostep S60. But if display setting is not established, then the controldevice 4 reaches a negative decision in step S50, and the flow ofcontrol is transferred to step S70.

In step S60, the control device 4 sends display information to thedisplay device 14 (refer to FIG. 1), and then the flow of controlproceeds to step S70. This display information may be, for example, amessage saying “stopped”, “doing emergency stop”, “turning right”, or“turning left” that is displayed upon the display device 14, accordingto 3 0 the information corresponding to the state of the vehicle 1determined during the traveling assistance setting processing (S30).

It should be understood that, instead of outputting such displayinformation, or along with outputting such display information, it wouldalso be acceptable to arrange to output an audio signal for replayingthe message described above to an audio replay device not shown in thefigures. In this case, it would also be acceptable to employ an audiodevice of a navigation device not shown in the figures as the audioreplay device not shown in the figures.

In step S70, the control device 4 makes a decision as to whether or notOFF actuation has been performed. Upon receipt of, for example, an OFFsignal from the vehicle 1 (for example, an OFF signal for the engine),the control device 4 reaches an affirmative decision in step S70 andperforms predetermined OFF processing, and then the processing in FIG.11 terminates. But if, for example, the control device 4 does notreceive an OFF signal from the vehicle 1, then it reaches a negativedecision in step S70 and the flow of control proceeds to step S80. Instep S80, the control device 4 waits for a predetermined time period(for example 0.1 seconds), and then the flow of control returns to stepS30. When the flow of control returns to step S30, the processingdescribed above is repeated.

The Initial Setting Processing

FIG. 12 is a flow chart for explanation of the details of step S20 ofthe FIG. 11 flow chart (i.e. the initial setting processing). In stepS21 of FIG. 12, the control device 4 inputs position information for thevehicle 1 from the GPS device 15 (refer to FIG. 1), and then the flow ofcontrol proceeds to step S22. In step S22, on the basis of the latitudeand longitude included in this position information, the control device4 sets a flag that shows whether the traffic lane along which thevehicle 1 is to travel is on the left or the right of the road, in otherwords Whether the vehicle travels along the left side or the right sideof the road. In concrete terms, the control device 4 determines the nameof the country in which the vehicle 1 is being used on the basis of thelatitude and the longitude. And, by reference to a database not shown inthe figures, the flag is set that shows whether driving on the roads inthat country is on the left side or on the right side. This databasethat specifies the relationship between the country name and left orright side driving is stored in advance in the storage unit 4 b.

In step S23, the control device 4 sets a flag that indicates theposition where the steering wheel 10 is attached to the vehicle 1 (i.e.on the right thereof or on the left thereof), and then the flow ofcontrol proceeds to step S24. Information that specifies whether thisvehicle is right hand drive or left hand drive is stored in advance inthe storage unit 4 b as specification information for the vehicle 1. Instep S24, the control device 4 determines the initial setting value onthe basis of a table such as that shown by way of example in FIG. 13. Itshould be understood that the order of steps S21 and S23 could bechanged.

According to FIG. 13, four initial setting values from “1” to “4” areprepared, according to the combination of the position of attachment ofthe steering wheel in the vehicle 1 (i.e. whether the vehicle is righthand drive or left hand drive) and the position of the traffic lane onthe road (i.e. upon the right side or upon the left side). In the caseof a right hand drive steering wheel and driving on the left, theinitial setting value is “4”.

In step S25, the control device 4 sets an initial position for theregion of attention 82. This initial position of the region of attention82 is positioned according to the initial setting value. In concreteterms, the control device 4 takes the initial position of the region ofattention 82 as being (Xq1, Yq) when the initial setting value is “1”,takes the initial position of the region of attention 82 as being (Xq2,Yq) when the initial setting value is “2”, takes the initial position ofthe region of attention 82 as being (Xq3, Yq) when the initial settingvalue is “3”, and takes the initial position of the region of attention82 as being (Xq4, Yq) when the initial setting value is “4”.

In this explanation, in the coordinate system that specifies the imagingregion 81, the position of the region of attention 82 is given by thecoordinates (Xq, Yq) of the center of the region of attention 82. FIG.10 shows an example of the region of attention when the initial settingvalue is “4”, and, since this is the case of a right hand drive steeringwheel and driving on the left, the initial position (Xq4, Yq) isdetermined so that the region of attention 82 is set toward the driver'sseat side (i.e. toward the right).in the left side traffic lane.

FIG. 14 shows an example of the region of attention 82 when the initialsetting value is “1”, and, since this is the case of a left hand drivesteering wheel and driving on the right, the initial position (Xq1, Yq)is determined so that the region of attention 82 is set toward thedriver's seat side (i.e. toward the left) in the right side trafficlane.

FIG. 15 shows an example of the region of attention 82 when the initialsetting value is “3”, and, since this is the case of a left hand drivesteering wheel and driving on the left, the initial position (Xq3, Yq)is determined so that the region of attention 82 is set toward thedriver's seat side (i.e. toward the left) in the left side traffic lane.

And 16 shows an example of the region of attention 82 when the initialsetting value is “2”, and, since this is the case of a right hand drivesteering wheel and driving on the right, the initial position (Xq2, Yq)is determined so that the region of attention 82 is set toward thedriver's seat side (i.e. toward the right) in the right side trafficlane.

In step S26 of FIG. 12, the control device sets the initial size of theregion of attention 82. In this embodiment, the initial size of theregion of attention 82 (i.e. Px (in the X axis direction)×Py(in the Yaxis direction)) is determined on the basis of the size of an object(for example, the size of the leading vehicle in front). If the leadingvehicle is included in the image acquired by the camera 3, then thecontrol device 4 estimates the size of the leading vehicle on the basisof the height of the image of the leading vehicle that has been capturedby the image capture chip 113, the focal length of the image captureoptical system 31 Which is already known, and the distance L from thevehicle 1 to the leading vehicle that has been obtained by rangefinding. And, the numbers of pixels in the image obtained upon theimaging chip 113 (i.e. Px (in the X axis direction)×Py(in the Y axisdirection)) by imaging a leading vehicle with an estimated size (forexample, width 3 (m)×height 1.4 (m)) from its rear 1 (m) away are takenas being its initial size.

Px and Py are calculated according to the following Equations (1) and(2):

Px=ox×L   (1)

Py=oy×L   (2)

Here, ox is the number of pixels in the X axis direction in the image ofthe leading vehicle L (m) away that has been captured by the imagecapture chip 113. And oy is the number of pixels in the Y axis directionin the image of the leading vehicle L (m) away that has been captured bythe image capture chip 113. L is the inter-vehicle distance from thevehicle 1 to the leading vehicle.

It should be understood that, in the coordinate system that specifiesthe imaging region 81, the value Yq described above that specifies theinitial position corresponds to the center of the height of the imageobtained upon the image capture chip (in this example, a location at aheight of 0.7 (m) on the leading vehicle) by imaging the above describedleading vehicle from 1 (m) away.

In step S27, the control device 4 outputs display information to thedisplay device 14 (refer to FIG. 1) and sets the flag a to 1, and thenthe processing of FIG. 12 terminates. This display information isinformation that specifies the end of the initial setting processing:for example, the message “initial setting completed” is displayed uponthe display device 14.

The Traveling Assistance Setting Processing

FIG. 17 is a flaw chart for explanation of the details of the travelingassistance setting processing. If, in step S310 of FIG. 17, theinformation specifying the position of the shift lever inputted from theshift lever position detection device 16 (refer to FIG. 1) is “P” (i.e.parking), then the control device 4 reaches an affirmative decision instep S310, and the flow of control proceeds to step S320. But if theinformation specifying the position of the shift lever inputted from theshift lever position detection device 16 (refer to FIG. 1) is not “P”,then the control device 4 reaches a negative decision in step S310, andthe flow of control is transferred to step S420. It should be understoodthat it would also be acceptable to arrange to apply the decision ofthis step S310 to the case when the shift lever is in “N” (i.e.neutral).

In step S320, the control device 4 inputs the vehicle speed V from thevehicle speed sensor 21, and then the flow of control proceeds to stepS330. The control device 4 may, for example, change the frame rate ofthe region of attention 82 according to the vehicle speed V. Asdescribed above, when the frame rate of the region of attention 82 isset to be higher as compared to the frame rate of the imaging region 81,then the control device 4 sets the frame rate of the region of attention82 to be higher as the vehicle speed V increases, and sets the framerate of the region of attention 82 to be lower as the vehicle speed Vdecreases. In this case, it would also be acceptable to arrange for thecontrol device 4 to apply control so as to make the frame rate of theimaging region 81 outside the region of attention 82 also beproportionate to the vehicle speed V. In step S330, the control device 4inputs the amount by which the brake pedal 8 a is being stepped uponfrom the brake control device 8 (refer to FIG. 1), and then the flow ofcontrol proceeds to step S340.

In step S340, on the basis of the vehicle speed V and the amount bywhich the brake pedal 8 a is being stepped upon (i.e. the angle throughwhich it is being depressed), the control device 4 makes a decision asto whether or not a flag Em=0. The flag Em is a flag that is set asshown by way of example in FIG. 18, on the basis of the vehicle speed Vand the amount of change of the amount by which the brake pedal 8 a isbeing stepped upon (i.e. the angle through Which it is being depressed).In this embodiment, it is determined that emergency braking (i.e. abruptbraking) is being performed if Em=1, while it is determined that normalbraking is being performed if Em=0. If Em=0 then the control device 4reaches an affirmative decision in step S340 and the flow of controlproceeds to step S350. But if Em=1 then the control device 4 reaches anegative decision in step S340 and the flow of control is transferred tostep S430.

It should be understood that, instead of the amount of change of theamount by which the brake pedal 8 a is being stepped upon (i.e. of theangle through which it is being depressed), it would also be acceptableto arrange to determine that Em=1 on the basis of the amount of changeof the opening amount of a brake valve not shown in the figures.Moreover, it would also be acceptable to arrange to determine that Em=1on the basis of the amount of change of the vehicle speed V, or toarrange to determine that Em=1 on the basis of the amount of change ofthe deceleration ratio of a speed change mechanism not shown in thefigures.

In step S350, the control device 4 inputs the amount by which theaccelerator pedal 7 a is being stepped upon from the throttle controldevice 7 (refer to FIG. 1), and then the flow of control proceeds tostep S360. And in step S360 the control device 4 inputs the rotationalangle θ of the steering wheel 10 from the steering control device 9, andthen the flow of control proceeds to step S370. In step S370, thecontrol device 4 makes a decision as to whether or not steeringactuation has been performed. If the rotational angle θ is greater thana predetermined value then the control device 4 reaches an affirmativedecision in step S370 and the flow of control proceeds to step S380,while if the rotational angle θ is less than or equal to thepredetermined value then a negative decision is reached in step S370 andthe flow of control is transferred to step S440.

In step S380, the control device 4 calculates the amount of shiftingX_(dist) of the region of attention 82 in the X axis direction on thebasis of the rotational angle θ of the steering wheel 10 and the vehiclespeed V, according to the following Equation (3):

X _(dist)=θ×(V×0.2)   (3)

According to Equation (3) above, the shifting amount X_(dist) becomesgreater, the greater is the steering angle of the steering system (inother words, the rotational angle θ of the steering wheel 10), and thegreater is the vehicle speed V.

In step S390, the control device 4 calculates the position (i.e. the Xcoordinate) of the region of attention 82 during traveling on the basisof the initial position (XqN, Yq) of the region of attention 82 that wasset during the initial setting processing, according to the followingEquation (4):

Xq=XqN+X _(dist)   (4)

Here, N is one of the initial setting values 1 through 4 that wasdetermined during the initial setting processing.

X_(dist) is the shifting amount of the region of attention 82 in the Xaxis direction as calculated in step S380, and corresponds to a numberof pixels in the X axis direction. Due to the processing in step S390,the position of the region of attention 82 changes according to steeringactuation. Moreover, the position of the region of attention 82 alsochanges according to the magnitude of the vehicle speed V.

In step S400, on the basis of the initial position (XqN, Yq) of theregion of attention 82 that was set in the initial setting processing,the control device 4 calculates the position (the Y coordinate) of theregion of attention 82 during traveling according to the followingEquation (5):

Yq=Yq+P(Z)   (5)

Here, P(Z) is the shifting amount of the region of attention 82 in the Yaxis direction, and is a number of pixels in the Y axis directioncorresponding to the depth Z (m). For example, this function may specifyto how many pixels in the Y axis direction an image of a road of depth20 (m) corresponds. The relationship P(Z) between the depth Z and thenumber of pixels is stored in advance in the storage unit 4 b (refer toFIG. 2).

In general, when the direction of travel along a flat straight road isimaged, the number of pixels in the Y axis direction corresponding tothe image of the road upon the image capture chip 113 increases as thedepth Z (m) from the vehicle 1 becomes deeper. Therefore, the value ofYq that corresponds to the center in height of the image obtained byimaging the above described leading vehicle from 1 (m) away increases asthe leading vehicle to which attention should being directed becomesfurther away (i.e. as the depth Z becomes deeper).

The control device 4 determines the depth Z of the leading vehicle towhich attention should be directed according to the following Equation(6):

Z=Za+Zb   (6)

Here, Za is the braking distance (m) upon a dry road, and Zb is thebraking distance (m) upon a wet road surface. Za and Zb may, forexample, be based on the values shown in FIG. 19. In this embodiment,the position of the region of attention 82 is determined so that theleading vehicle at the depth Z in front of the vehicle 1 (in otherwords, at the position separated by Z (m) from the vehicle 1) isincluded in the region of attention 82. This is based upon the idea ofdirecting attention further away than the distance that is required forstopping the vehicle if emergency braking is applied. Values (Za+Zb) ofthe depth Z corresponding to vehicle speeds V are stored in advance inthe storage unit 4 b (refer to FIG. 2). According to the processing ofstep S400, the position of the region of attention 82 changes accordingto change of the vehicle speed V.

For the region of attention 82 whose position has changed in thismanner, at least one of the frame rate, the gain, the decimation ratio,the charge accumulation time, and so on is changed between the imagingregion 81 and the region of attention 82.

In step S410, on the basis of the initial size (Px×Py) of the region ofattention 82 that was set in the initial setting processing, the size(X_wid, Y_wid) of the region of attention 82 during traveling iscalculated according to the following Equations (7) and (8), and thenthe processing of FIG. 17 terminates.

X_wid=Px/Z   (7)

Y_wid=Py/Z   (8)

Here, Px is the number of pixels in the X axis direction set in stepS26, and Py is the number of pixels in the Y axis direction set in stepS26. According to the above Equations (7) and (8), the size (X_wid,Y_wid) of the region of attention 82 during traveling becomes smallerthan the initial size (Px×Py) of the region of attention 82, the furtheraway the leading vehicle to which attention should be directed becomes(i.e. the deeper the depth Z becomes). According to the processing ofstep S410, the size of the region of attention 82 changes according tochange of the vehicle speed V.

For the region of attention 82 whose size has been changed in thismanner, at least one of the frame rate, the gain, the decimation ratio,the charge accumulation time, and so on is different between the imagingregion 81 and the region of attention 82.

In step S420 to which the flow of control is transferred when a negativedecision is reached in step S310 described above, the control device 4performs setting processing while stopped, and the processing of FIG. 17terminates. This setting processing while stopped determines theposition of the region of attention 82 so that a vehicle in front thatis separated by, for example, 1 (m) is included in the region ofattention 82. Moreover, the size of the X axis direction of the regionof attention 82 is set to a maximum, so that objects in positions closeto the side portions of the vehicle 1 are also, to the greatest possibleextent, included in the region of attention 82.

In step S430 to which the flow of control is transferred when a negativedecision is reached in step S340 described above, the control device 4performs setting processing during abrupt braking determination, andthen the processing of FIG. 17 terminates. In this setting processingduring abrupt braking determination, for example, in the region ofattention 82, decimation is stopped, the frame rate is raised tomaximum, the charge accumulation time is shortened, and the gain is setto be higher.

It should be understood that it would also be acceptable to arrange forthe control device 4 to increase the frame rate of the imaging region 81outside the region of attention 82. Moreover, the control device 4 mayissue a command to the camera 3 for performing recording, so as to storein the recording unit 36 the images acquired by the camera 3 during apredetermined time period (for example, 5 seconds to 15 seconds) after anegative decision has been reached in step S340.

After abrupt stopping, the control device 4 further shifts the region ofattention 82 to the initial position for the region of attention 82 thatwas set during the initial setting processing (step S25 of FIG. 12), andalso changes the size of the region of attention 82 to the initial size(Px×Py) of the region of attention 82 that was set during the initialsetting processing (step S26 of FIG. 12). Due to this, the position andthe size of the region of attention 82 that were changed due to thevehicle speed V during traveling are returned to a position and sizethat are appropriate when the vehicle is stopped.

In step S440 to which the flow of control is transferred when a negativedecision is reached in step S370 described above, the control device 4performs setting for not shifting the position of the region ofattention 82 (i.e. its X coordinate) during traveling. In other words,if the rotational angle θ of the steering wheel 10 is less than or equalto a predetermined value, θ is set to 0, and also the value of X_(dist)is set to 0. That is, if the actuation angle of the steering wheel 10 isless than the predetermined value, the position of the region ofattention 82 (i.e. its X coordinate) is preserved unchanged. Due tothis, it becomes easier to alleviate the processing burden during finesteering actuation which is not actuation for turning.

FIG. 20(a) is a figure showing an example of shifting of the position ofthe region of attention 82 and change of the size of the region ofattention 82 when a right turn is to be made on a normal road at anintersection. According to the traveling assistance setting processingdescribed above, if the vehicle 1 is waiting behind a leading vehicle infront in order to perform a right turn, the region of attention 82A isits initial position, and the size of the region of attention 82A isapproximately the same as its initial size (Px×Py). When, in the statein which the vehicle 1 is moving forward, the driver starts to actuatethe steering toward the right direction, then the position of the regionof attention 82B shifts slantingly rightward and upward. And, since thevehicle speed V is low, accordingly the size of the region of attention82A is also approximately the same as its initial size (Px×Py)

And FIG. 20(b) is a figure showing an example of shifting of theposition of he region of attention 82 and change of the size of theregion of attention 82 when changing vehicle lane and accelerating intothe overtaking vehicle lane on the right side of a high speed road.According to the traveling assistance setting processing describedabove, if the vehicle 1 is traveling at high speed, then the position ofthe region of attention 82A is higher than its initial position, and thesize of the region of attention 82A is smaller as compared to itsinitial size (Px×Py). When, in the state in which the vehicle 1 isaccelerating, the driver actuates the steering toward the rightdirection, then the position of the region of attention 82B shiftsslantingly rightward and upward. And, since the vehicle speed is high,accordingly the size of the region of attention 82A is yet smaller, itshould be understood that FIG. 20 is an example when driving on the leftside, and these figures could also be employed with appropriatemodification in the case of turning left when driving on the right side,or in the case of changing vehicle lane when driving on the right side.Moreover, it would also be acceptable to arrange to detect the sightline of the driver with the use of a sight line detection device notshown in the figures (for example, a sight line detection device on thesteering wheel), and to set a region at which the driver is not looking,or a region that constitutes a blind area, as the region of attention82.

It is to be understood that, for the above sight line detection, the useof any sight line detection method would be acceptable, such as acorneal reflection method in which the direction of sight line of thedriver is detected by reflecting infra-red rays from his cornea, or alimbus tracking method in which the difference in light reflectancebetween the cornea and the sclera is employed, or an image analysismethod of detecting the sight line of the driver by picking up an imageof his eyeball with a camera and performing image processing thereupon,or the like.

According to this first embodiment, the following beneficial operationaleffects are obtained.

(1) Since there are provided the control device 4 that recognizes atleast one of the specifications of the vehicle 1 to which the system ismounted and operations upon the actuation unit of the vehicle 1, theimage capture unit 32, that has at least the region of attention 82 andthe imaging region 81 and that captures the exterior of the vehicle 1,and the control device 4 that sets the image capture conditions of theregion of attention 82 and the image capture conditions of the imagingregion 81 to be different on the basis of the results of recognition bythe control device 4, accordingly it is possible to set the imagecapture conditions for the camera 3 in an appropriate manner.

(2) Since the control device 4 recognizes the position where thesteering wheel 10 is attached to the vehicle (i.e. on its right or onits left side), accordingly it is possible to set the image captureconditions for the camera 3 according to the position in which thedriver is riding in the vehicle.

(3) Since the setting unit sets the frame rate for the region ofattention 82 and the frame rate for the imaging region 81 to bedifferent according to the position of the steering wheel 10,accordingly it is possible to increase the frame rate for a region ofattention 82 that is, for example, on the side of the driver's seat(i.e. on the right) or the like, so that it is possible to set the imagecapture conditions for the camera 3 in an appropriate manner.

(4) Since the control device 4 is provided that detects informationrelated to the vehicle speed V of the vehicle 1, and this control device4 sets the image capture conditions for the region of attention 82 andthe image capture conditions for the imaging region 81 to be differentaccording to the result of detection of information related to thevehicle speed V, accordingly it is possible to set the image captureconditions of the camera 3 in an appropriate manner according to thevehicle speed V.

(5) Since, when the information related to the vehicle speed V increasesand when it decreases, the control device 4 varies at least one of theimage capture conditions for the region of attention 82 and the-imagecapture conditions for the imaging region 81, accordingly it is possibleto set the image capture conditions of the camera 3 in an appropriatemanner by, for example, making the frame rate higher, the faster is thevehicle speed V.

(6) Since, when the rotational angle θ of the steering wheel 10 exceedsthe predetermined value, the control device 4 varies at least one of theframe rate for image capture in the region of attention 82 and the framerate for image capture in the imaging region 81 by increasing it,accordingly it is possible to vary the image capture conditions of thecamera 3 during turning operation.

(7) Since the control device 4 that transmits display information to thedisplay device 14 of the vehicle 1 on the basis of the results of imagecapture by the image capture unit 32 is provided, accordingly it ispossible to provide necessary information to someone who is riding inthe vehicle 1.

(8) Since, when the rotational angle 0 of the steering wheel 10 does notreach the predetermined value, the control device 4 maintains thesetting of at least one of the frame rate for image capture in theregion of attention 82 and the frame rate for image capture in theimaging region 81, accordingly it is possible to avoid changing theimage capture conditions during minute steering actuation when thevehicle is not actually turning. Due to this, for example, it ispossible to prevent the frame rate of the region of attention 82 frombeing minutely changed more often than necessary, and this is helpfulfor alleviating the processing burden.

(9) Since the control device 4 includes at least one of frame rate ofimage capture, gain, decimation, pixel signal addition, chargeaccumulation, bit length, size of the imaging region, and position ofthe imaging region in the image capture conditions that are to bedifferent between the image capture condition for the region ofattention 82 and the image capture condition for the imaging region 81,accordingly it is possible to set the image capture conditions of thecamera 3 in an appropriate manner.

(10) Since the control device 4 changes at least one of the centerposition of the region of attention 82 and the center position of theimaging region 81 on the basis of the result of detection of informationrelated to the vehicle speed V, accordingly it is possible to set theimage capture conditions of the camera 3 in an appropriate manner, suchas to change the position of the region of attention 82 along withchange of the vehicle speed V or the like.

(11) Since the control device 4 changes at least one of the size of theregion of attention 82 and the size of the imaging region 81 on thebasis of the result of detection of information related to the vehiclespeed V, accordingly it is possible to set the image capture conditionsof the camera 3 in an appropriate manner, such as to change the size ofthe region of attention 82 along with change of the vehicle speed V orthe like.

(12) Since the control device 4 sets the imaging region 81 to surroundthe region of attention 82, accordingly it is possible to set the imagecapture conditions of the camera 3 in an appropriate manner.

(13) Since the steering wheel 10 is provided and serves as an actuationunit of the vehicle 1, and since the control device 4 changes at leastone of the center position of the region of attention 82 and the centerposition of the imaging region 81 on the basis of actuation of thesteering wheel, accordingly it is possible to set the image captureconditions of the camera 3 in an appropriate manner, such as to changethe position of the region of attention 82 along with change of thecourse of the vehicle 1 or the like.

It should be understood that while, in the embodiment described above,the camera 3 was controlled under the control of the control device 4,it would also be acceptable to arrange for a part of the control of thecamera 3 to be performed by the control unit 35 of the camera 3.

One or a plurality of variant embodiments such as the following may alsobe combined with the first embodiment described above,

Variant Embodiment #1

In the traveling assistance setting processing, it would also beacceptable to arrange for the control device 4 to change the position ofthe region of attention 82 and the size of the region of attention 82according to an actuation signal from the turn signal switch 11. Asshown by way of example in FIG. 21, the control device 4 may change thesize of the region of attention 82 and/or set the image captureconditions for the region of attention 82 on the basis of the initialsetting value that was determined in step S24, and the direction of theturn signal resulting from actuation of the turn signal switch 11.

For example, to explain with reference to FIG. 10, when the steeringwheel is on the right side of the vehicle and the vehicle is driving onthe left side of the road so that the initial setting value is “4”, andif the turn signal is indicating the left direction, then the controldevice 4 performs control so as to include the left edge of the road inthe region of attention 82. In concrete terms, the control device 4widens the region of attention 82 of FIG. 10 toward the left side. Thiswidening of the region of attention 82 toward the left side is in orderto prevent involvement in an accident while turning left. Conversely, ifthe turn signal is indicating the right direction, then the controldevice 4 performs control so as to include the opposite vehicle lane inthe region of attention 82. In concrete terms, the control device 4widens the region of attention 82 of FIG. 10 toward the right side.

To explain with reference to FIG. 14, when the steering wheel is on theleft side of the vehicle and the vehicle is driving on the right side ofthe road so that the initial setting value is “1”, and if the turnsignal is indicating the left direction, then the control device 4performs control so as to include the opposite vehicle lane in theregion of attention 82. In concrete terms, the control device 4 widensthe region of attention 82 of FIG. 14 toward the left side. Conversely,if the turn signal is indicating the right direction, then the controldevice 4 performs control so as to include the right edge of the road inthe region of attention. 82. In concrete terms, the control device 4widens the region of attention 82 of FIG. 14 toward the right side. Thiswidening toward the right side is in order to prevent involvement in anaccident while turning right.

To explain with reference to FIG. 16, when the steering wheel is on theright side of the vehicle and the vehicle is driving on the right sideof the road so that the initial setting value is “2”, and if the turnsignal is indicating the left direction, then the control device 4performs control so as to include the opposite vehicle lane in theregion of attention 82. In concrete terms, the control device 4 greatlywidens the region of attention 82 of FIG. 16 toward the left side.Conversely, if the turn signal is indicating the right direction, thenthe control device 4 performs control so as to include the right edge ofthe road in the region of attention 82. In concrete terms, the controldevice 4 somewhat widens the region of attention 82 of FIG. 16 towardthe right side. This widening toward the right side is in order toprevent involvement in an accident while turning right.

And to explain with reference to FIG. 15, when the steering wheel is onthe left side of the vehicle and the vehicle is driving on the left sideof the road so that the initial setting value is “3”, and if the turnsignal is indicating the left direction, then the control device 4performs control so as to include the left edge of the road in theregion of attention 82. In concrete terms, the control device 4 somewhatwidens the region of attention 82 of FIG. 15 toward the left side. Thiswidening toward the left side is in order to prevent involvement in anaccident while turning left. Conversely, if the turn signal isindicating the right direction, then the control device 4 performscontrol so as to include the opposite vehicle lane in the region ofattention 82. In concrete terms, the control device 4 greatly widens theregion of attention 82 of FIG. 15 toward the right side.

FIG. 22 is a flow chart for explanation of the processing according tothis Variant Embodiment #1 when the turn signal switch 11 is actuated.During traveling assistance setting processing, the control device 4starts the processing of FIG. 22 as a subroutine when an actuationsignal from the turn signal switch 11 is inputted. In step S510 of FIG.22 the control device 4 makes a decision as to whether or not thedirection of the turn signal is toward the left. If the turn signaldirection is toward the left, then the control device 4 reaches anaffirmative decision in step S510 and the flow of control proceeds tostep S520, while if the turn signal direction is toward the right, thenthe control device 4 reaches a negative decision in step S510 and theflow of control is transferred to step S530.

In step S520 the control device 4 makes a decision as to whether or notthe position of the traffic lane is to the left side. If the vehicle istraveling on the left side, then the control device 4 reaches anaffirmative decision in step S520 and the flow of control is transferredto step S550, while if the vehicle is traveling on the right side, thenthe control device 4 reaches a negative decision in step S520 and theflow of control is transferred to step S540.

In step S540 the control device 4 controls the image capture unit 32 sothat the opposite vehicle lane is included in the region of attention82, and then the processing of FIG. 22 terminates. And in step S550 thecontrol device 4 controls the image capture unit 32 so that the leftedge of the road is included in the region of attention 82, and then theprocessing of FIG. 22 terminates.

In step S530 the control device makes a decision as to whether or notthe position of the traffic lane is to the left side. If the vehicle istraveling on the left side, then the control device 4 reaches anaffirmative decision in step S530 and the flow of control is transferredto step S560, while if the vehicle is traveling on the right side, thenthe control device 4 reaches a negative decision in step S530 and theflow of control is transferred to step S570.

In step S560 the control device 4 controls the image capture unit 32 sothat the opposite vehicle lane is included in the region of attention82, and then the processing of FIG. 22 terminates. And in step S570 thecontrol device 4 controls the image capture unit 32 so that the rightedge of the road is included in the region of attention 82, and then theprocessing of FIG. 22 terminates.

It should be understood that, if the turn signal switch 11 is turned OFFafter the processing of FIG. 22 has been executed, then the controldevice 4 cancels the changing of the size of the region of attention 82due to FIG. 22. Moreover, if the turn signal switch 11 is turned ON,then, even if the vehicle speed V is zero, as compared to the imagingregion 81, settings are made to raise the frame rate for the region ofattention 82, to increase the gain, to lower the decimation ratio, andto shorten the charge accumulation time. However, if at least one of theframe rate, the gain, the decimation ratio, the charge accumulationtime, or the like is to be made to be different between the imagingregion 81 and the region of attention 82, then only the image captureconditions that are to be different are changed.

Since, according to Variant Embodiment #1 as explained above, settingsare established according to actuation of the turn signal switch 11 inorder to make the image capture conditions for the region of attention82 and the image capture conditions for the imaging region 81 to bedifferent, accordingly, if for example the vehicle is turning right orleft at an intersection, then the opposite vehicle lane is included inthe region of attention 82 so that it is possible reliably to detectoncoming vehicles, and/or the road edge is included in the region ofattention 82 so that it is possible to prevent involvement in anaccident, and as a result it is possible to set the region of attention82 in an appropriate manner. Furthermore, the frame rate for the regionof attention 82 is set to be higher as compared to that for the imagingregion 81 and so on, so that it is possible to set the image captureconditions for the imaging region 81 and for the region of attention 82in an appropriate manner.

Variant Embodiment #2

It would also be acceptable to provide a structure in which, in thetraveling assistance setting processing, the control device 4 changesthe position of the region of attention 82 and the size of the region ofattention 82 according to change in the distance between an object suchas a bicycle, an ordinary vehicle, a large sized vehicle, a pedestrianor the like, and the vehicle 1.

In Variant Embodiment #2, when, for example, the vehicle 1 gets nearerto a leading vehicle in front so that the distance L to the leadingvehicle (i.e. the inter-vehicle distance) becomes shorter, the controldevice 4 determines the position of the region of attention 82 so as toinclude the leading vehicle in the region of attention 82. Here, changeof the inter-vehicle distance L from the vehicle 1 to the leadingvehicle is obtained on the basis of the images acquired by the camera 3at predetermined time intervals synchronized with the timing ofacquisition of the vehicle speed V by the range finding calculation unit35 a of the camera 3 repeatedly range finding the distance L (i.e. theinter-vehicle distance) to the leading vehicle within the photographicscreen.

The control device 4 calculates the position (i.e. the Y coordinate) ofthe region of attention 82 during travel by using the inter-vehicledistance L, instead of the depth Z in the above Equation (5). Due tothis, in the ease of picking up an image of a leading vehicle in fronton a flat straight road, the value of Yq that specifies the position ofthe region of attention 82 in the Y axis direction upon the imagecapture chip 113 increases as the inter-vehicle distance L becomeslonger, and decreases as the inter-vehicle distance L becomes shorter.

Furthermore, since the leading vehicle is photographed by the camera 3as being larger when the inter-vehicle distance L changes to becomeshorter, accordingly the control device 4 sets the size of the region ofattention 82 to be larger. Conversely, since the leading vehicle isphotographed by the camera 3 as being smaller when the inter-vehicledistance L changes to become longer, accordingly the control device 4sets the size of the region of attention 82 to be smaller. The controldevice 4 calculates the size of the region of attention 82 duringtraveling by substituting the inter-vehicle distance L into Equations(7) and (8) above, instead of the depth Z.

While according to the processing of FIG. 17 described above the size ofthe region of attention 82 is set to be smaller when the vehicle speed Vbecomes faster, in this Variant Embodiment #2, if the inter-vehicledistance L to a leading vehicle in front is short, the size of theregion of attention 82 is set to be large even if the vehicle speed V ishigh, and accordingly it is possible to include the leading vehicle inthe region of attention 82 in an appropriate manner. Due to this,detection of change of the traveling state of the leading vehicle on thebasis of the images acquired by the camera 3 becomes simpler and easier,as compared to the case when the size of the region of attention 82continues to be set to be small.

Also for the region of attention 82 whose size and position are changedin this manner, it will be acceptable to change at least one of theframe rate, the gain, the decimation ratio, the charge accumulationtime, or the like between the imaging region 81 and the region ofattention 82.

Variant Embodiment #3

If an object such as a bicycle, an ordinary vehicle, a large sizedvehicle, a pedestrian or the like is detected in the vicinity of thevehicle 1, even outside the previously set region of attention 82, itwould also be acceptable for the control device 4 to set the region ofattention 82 anew so as to include this object in Variant Embodiment #3,when an object that has been detected has moved, the control device 4sets the region of attention 82 anew so as to include this object. Forexample, if the distance between the object that has been detected andthe vehicle 1 has become closer and is within a predetermined distance,then the control device 4 may set the region of attention 82 anew so asto include this object. And, if the distance between the object that hasbeen detected and the vehicle 1 has become further away and is greaterthan or equal to the predetermined distance, then the control device 4may cancel the setting of the region of attention 82 that includes thisobject.

Also for the region of attention 82 whose size and position have beenset in this manner, it will be acceptable for at least one of the framerate, the gain, the decimation ratio, the charge accumulation time, orthe like to be made to be different between the imaging region 81 andthe region of attention 82.

Since, according to this Variant Embodiment #3, the control device 4 isprovided to detect moving objects in the surroundings of the vehicle 1on the basis of the information from the camera 3, and since thiscontrol device changes at least one of the image capture conditions forthe region of attention 82 or the image capture conditions for theimaging region 81 on the basis of the result of this detection of movingobjects, accordingly it is possible to set the image capture conditionsfor the camera 3 in an appropriate manner according to the presence orabsence of moving objects.

Furthermore since, when on the basis of information from the camera 3 ithas been detected that the distance between the vehicle 1 and a movingobject has become closer and is within the predetermined distance, thecontrol device 4 changes at least one of the frame rate for imagecapture of the region of attention 82 and the frame rate for imagecapture of the imaging region 81 in the increase direction, accordinglydetection of the change of the movement state of the moving object onthe basis of the images acquired by the camera 3 becomes simple andeasy.

Variant Embodiment #4

It would also be acceptable to set the region of attention 82 a new onthe basis of the color of the image acquired by the camera 3. Thecontrol device 4 may set a region in the image that includes a redcolored object to be the region of attention 82. By adding this regionthat includes a red colored object to the region of attention 82, it ispossible to include, for example, a red light signal, an alarm on arailroad crossing, a red lamp on an emergency vehicle, or the likewithin the region of attention 82.

Also for the region of attention 82 that has been set in this manner, itwill be acceptable for at least one of the frame rate, the gain, thedecimation ratio, the charge accumulation time, or the like to be madeto be different between the imaging region 81 and the region ofattention 82.

Variant Embodiment #5

It would also be acceptable to set the region of attention 82 a new onthe basis of sound information in the sound captured by the microphone17 of the vehicle 1. If for example a level of sound information on theright side of the vehicle 1 that is greater than a predetermined valueis inputted, then the control device 4 may widen the region of attention82 toward the right side in the imaging region 81, or may reset theregion of attention 82 toward the right side in the imaging region 81.This provision of the region of attention 82 toward the right side is inorder to collect information relating to conditions outside the vehicle1 on its right side.

Moreover, if for example a level of sound information on the left sideof the vehicle 1 that is greater than a predetermined value is inputted,then the control device 4 may widen the region of attention 82 towardthe left side in the imaging region 81, or may reset the region ofattention 82 toward the left side in the imaging region 81. Thisprovision of the region of attention 82 toward the left side is in orderto collect information relating to conditions outside the vehicle 1 onits left side.

Also for the region of attention 82 that has been set in this manner, itwill be acceptable for at least one of the frame rate, the gain, thedecimation ratio, the charge accumulation time, or the like to be madeto be different between the imaging region 81 and the region ofattention 82.

Variant Embodiment #6

While, in the first embodiment, a case in which a region of attention 82that includes a leading vehicle was set, it would also be acceptable toarrange to set a region of attention 82 that includes a vehicle oncomingtoward the vehicle 1. In this Variant Embodiment #6, from among theobjects that are present within the travel region described above andthat are traveling in the opposite direction (i.e. that are oncoming tothe vehicle 1), the control device 4 will recognize the vehicle that isclosest to the vehicle 1 as being an oncoming vehicle.

In the image acquired by the camera 3, the control device 4 sets theregion that corresponds to the position of the oncoming vehicledescribed above as being the region of attention 82. The control device4 may particularly set the region of attention 82 to include the numberplate of the oncoming vehicle, and/or to include the face of the driverof the oncoming vehicle who is sitting in the driver's seat and isdriving.

By setting a region that includes the number plate of the oncomingvehicle or the face of the driver of the oncoming vehicle as the regionof attention 82, it is possible to include the oncoming vehicle that isapproaching the vehicle 1 in the region of attention 82 in anappropriate manner.

Also for the region of attention 82 that has been set in this manner, itwill be acceptable for at least one of the frame rate, the gain, thedecimation ratio, the charge accumulation time, or the like to be madeto be different between the imaging region 81 and the region ofattention 82.

Variant Embodiment #7

While, in the first embodiment, an example was explained in which theimaging region 81 included the region of attention 82 (i.e. surroundedthe region of attention 82), it would also be acceptable to arrange toset the imaging region 81 and the region of attention 82 to be side byside, left and right. It would then be possible to vary the sizes andthe positions of the imaging region 81 and the region of attention 82 bymoving the boundary line between the imaging region 81 and the region ofattention 82 left or right. Also with a region of attention 82 that hasbeen set in this manner, it will be acceptable for at least one of theframe rate, the gain, the decimation ratio, the charge accumulationtime, or the like to be made to be different between the imaging region81 and the region of attention 82.

And while, in the first embodiment, as distance measurement performed bythe camera 3, a technique was employed for performing range findingcalculation by using image signals from pixels for focus detection thatwere provided upon the imaging element 100, it would also be possible toemploy a technique for performing distance measurement by using twoimages from a stereo camera. Moreover, it would also be possible toemploy a technique for performing distance measurement by using amillimeter wave radar separate from the camera 3.

Embodiment #2

In this embodiment, in order to make it easy to find a line upon theroad when the traveling environment changes, as for example whentraveling in a tunnel or during rainfall or the like, chargeaccumulation control for the imaging element 100 (i.e. of the imagecapture chip 113 (refer to FIG. 3)) of the camera 3 (refer to FIGS. 1and 9) is performed independently for each of the block units describedabove.

In addition to detecting objects as explained in the first embodiment,the control device 4 also performs detection as described below. Thatis, from the image data acquired by the camera 3, the control device 4performs shape detection, color detection, and brightness detection ofobjects included in the image. In concrete terms, the control device 4detects data for lines such as white lines or guard rails or the likethat are provided along the road. Moreover, on the basis of thebrightness situation for each unit region 131 of the image capture chip113, the control device 4 detects tail lamp (tail light) datarepresenting the state of lighting of the leading vehicle in front.

Furthermore, in this embodiment, the beam changeover switch 18 and therainfall sensor 19 of FIG. 1 are employed, although they were notutilized in the first embodiment. The beam change over switch 18 is anactuation member for changing over the angle of illumination of thelighting equipment (i.e. the headlights) of the vehicle between at leasttwo stages in the vertical direction. For example, the beam change overswitch may change over the headlamp beams between shining in anapproximately horizontal direction (i.e. “high beam”) and shining in adirection lower than the horizontal direction (i.e. “low beam”). Theactuation signal from the beam change over switch 18 is sent both to thelighting equipment not shown in the figure, and to the control device 4.

The rainfall sensor 19 is a detector that detects droplets of rain by anoptical method or by an electrostatic capacitance method, and isattached inside the vehicle or outside the vehicle. The detection signalfrom the rainfall sensor 19 (i.e. rainfall information) is sent to thecontrol device 4.

FIG. 23 is a figure schematically showing an image of a photographicsubject (i.e. of an object) that is focused upon the image capture chip113. While actually an inverted image is focused, this is shown in thefigure as an erect image, for the convenience of understanding. In FIG.23, the vehicle 1 is traveling behind a leading vehicle 84 that is infront. Images of white lines 82 a, 82 b, and 82 c that are provided uponthe road, of the leading vehicle 84 in front, and of an oncoming vehicle85 are included upon the imaging surface 70 of the image capture chip113 (i.e. in its photographic region). Among these objects, the whitelines 82 a and 82 b that indicate the boundaries of the vehicle trafficlane upon which the vehicle 1 is traveling (i.e. the lane of travel) areobjects that are particularly important during detection of the path oftravel by the control device 4. Accordingly, in this explanation, animaging region that includes the white lines 82 a and 82 b is termed the“region of attention 71”.

It should be understood that there are some reference numerals in FIGS.23, 27, 29, 31, 33 and so on that are the same as ones used in thepreceding drawings, but these reference numerals apply to theexplanation of the second embodiment.

The control device 4 sets different conditions for the region ofattention 71 upon the imaging surface 70 and for the region 72 outsidethat region (i.e. the region to which particular attention is notdrawn), so as to perform charge accumulation (i.e. image capture). Here,the sizes and positions of the region of attention 71 and the regionwithout particular attention 72 upon the imaging surface 70 of the imagecapture chip 113 are included in the image capture conditions.

Along with setting first conditions and controlling image capture forthe unit regions 131 (refer to FIG. 4) that are included in the regionof attention 71, the control device 4 also sets second conditions andcontrols image capture for the unit regions 131 that are included in theregion without particular attention 72.

It should be understood that it would also be acceptable to provide aplurality of regions of attention 71 and/or a plurality of regionswithout particular attention 72, and also it would be acceptable toprovide, within the region of attention 71 or within the region withoutparticular attention 72, a plurality of regions for which the chargeaccumulation control conditions (i.e. the image capture conditions) aredifferent. Furthermore, in the row direction and the column direction ofthe imaging surface 70, it would also be acceptable to provide one ormore inactive regions in which charge accumulation(i.e. image capture)is not performed.

Explanation of the Flow Charts

The control procedure for the camera 3 and the setting of the imagecapture conditions will now be mainly explained in the following withreference to flow charts thereof (FIGS. 24 and 25). FIG. 24 is a flowchart for explanation of the overall flow of a control procedure for thecamera 3, executed by the control device 4. A program for executing theprocessing of the flow chart of FIG. 24 is stored in the storage unit 4b of the control device 4. The control device 4 may start this programthat performs the processing of FIG. 24, for example, when the supply ofpower from the vehicle 1 is started (i.e. when the system goes ON), orwhen the engine is started.

In step S10 of FIG. 24, the control device 4 makes a decision as towhether or not a flag a is equal to zero. The flag a is a flag that isset to 1 if the initial setting has been completed, and that is set tozero if the initial setting has not yet been completed. If the flag a iszero then the control device reaches an affirmative decision in step S10and the flow of control proceeds to step S20, while if the flag a is notzero then a negative decision is reached in step S10 and the flow ofcontrol is transferred to step S30.

In step S20, the control device 4 performs initial setting of the camera3, and then the flow of control proceeds to step S30. This initialsetting is setting that is determined in advance for causing the camera3 to perform predetermined operation. Due to this, the camera 3 sets thesame image capture conditions for the entire area of the imaging surfaceof the imaging element 100; for example, the camera 3 may start imagecapture at a frame rate of 60 frames per second (i.e.60 fps).

In step S30, the control device 4 performs image capture conditionsetting processing, and then the flow of control proceeds to step S40.This image capture condition setting processing is processing in Which aregion of attention 71 (refer to FIG. 23) and a region withoutparticular attention 72 outside it (also refer to FIG. 23) are set uponthe imaging element 100, and image capture conditions are determined foreach of these. The details of this image capture condition settingprocessing will be described hereinafter. In this embodiment, for theregion of attention 71, as compared to the region without particularattention 72, the frame rate is set to be higher, the gain is set to behigher, the decimation ratio is set to be lower, and the chargeaccumulation time is set to be shorter. The camera 3 performs imagecapture on the basis of these settings, and also performs the distancemeasurement (i.e. the range finding) described above.

It should be understood that it is not necessary for all of the framerate, the gain, the decimation ratio, the charge accumulation time, andso on to be different between the region of attention 71 and the regionwithout particular attention 72; it will be acceptable if even only atleast one of these parameters is different.

in step S40 of FIG. 24, the control device 4 acquires image data andrange finding data acquired by the camera 3 alter the image capturecondition setting processing, and also acquires information from varioussections within the vehicle 1, and then the flow of control proceeds tostep S50. In step S50, the control device 4 makes a decision as towhether or not setting for display of this information has been made. Ifsetting for display has been made, then the control device 4 reaches anaffirmative decision in step S50 and the flow of control proceeds tostep S60. But if setting for display has not been made, then the controldevice 4 reaches a negative decision in step S50 and the flow of controlis transferred to step S70.

In step S60, the control device 4 outputs display information to thedisplay device 14 (refer to FIG. 1), and then the flow of controlproceeds to step S70. This display information is informationcorresponding to the state of the vehicle 1 as determined during theimage capture condition setting processing (in step S30); for example,messages such as “entering a tunnel” or “headlights have been turned on”may be displayed upon the display device 14.

It should be understood that, instead of outputting the displayinformation, or along with outputting the display information, it wouldalso be acceptable to arrange to output an audio signal for replayingthe message described above via an audio replay device not shown in thefigure. In this case as well, as the audio replay device not shown inthe figures, it would be acceptable to use an audio device of anavigation device, not shown in the figures.

In step S70, the control device 4 makes a decision as to whether or notOFF actuation has been performed. When, for example, an OFF signal (forexample, a system OFF signal or an engine OFF signal) is received fromthe vehicle 1, the control device 4 reaches an affirmative decision instep S70 and performs predetermined OFF processing, and then theprocessing of FIG. 24 terminates. But if, for example, the controldevice 4 does not receive an OFF signal from the vehicle 1, then anegative decision is reached in step S70 and the flow of control returnsto step S30. Upon return to step S30, the processing described above isrepeated.

Image Capture Condition Setting Processing

The details of the image capture condition setting processing (in stepS30) will now be explained with reference to the flow chart of FIG. 25.In this embodiment, 5 examples are shown in which change of thetraveling environment (i.e. of the road state) of the vehicle 1 occurs,and the image capture conditions related to the region of attention andthe region without particular attention are determined.

In step S31 of FIG. 25, the control device 4 performs the imageprocessing described above upon the image data acquired by the camera 3,and then the flow of control proceeds to step S33. In step S33, thecontrol device 4 detects the white line data described above from theimage data, and then the flow of control proceeds to step S35. Asdescribed above, the control device 4 takes a region that includes thewhite line 82 a and the white line 82 b within the photographic region70 as being the region of attention 71. To explain this with referenceto FIG. 23, the region (shown by the broken line) having a trapezoidalshape and including three elements, i.e. the white line 82 a, the whiteline 82 b, and the region between the white line 82 a and the white line82 b is taken as the region of attention 71.

In step S35, the control device 4 takes the region of attention 71described above as a first imaging region 71, and takes the region otherthan this first imaging region 71 as a second imaging region 72, andthen the flow of control proceeds to step S37.

In step S37, the control device 4 sends commands to the camera 3, andsets the frame rate for the first imaging region 71 to be higher thanthe frame rate for the second imaging region 72. For example, the framerate for the first imaging region 71 may be set to 120 frames per second(120 fps), while the frame rate for the second imaging region 72 is setto 60 frames per second (60 fps). This is done in order to enhance thefrequency at which information is acquired related to the white lines,which are the objects to which attention should particularly be directedwhile traveling. The processing up to step S37 explained above is anexample of processing in a normal traveling environment (for example,traveling during the day in sunny weather).

If the traveling environment is different from the normal one, then thecontrol device 4 changes the image capture conditions for the region ofattention and for the region without particular attention describedabove. FIG. 26 is a flow chart showing an example of processing uponchange to a first traveling environment (i.e. to a first road. state).And FIG. 27 is a figure schematically showing an image of a photographicsubject (i.e. of objects) that is imaged upon the image capture chip 113when the vehicle 1 is approaching the entry of a tunnel 83.

In step S110 of FIG. 25, the control device 4 decides whether or not atunnel is present. If an entry to or an exit from a tunnel is includedin the image acquired by the camera 3, then the control device 4 reachesan affirmative decision in step S110, and the flow of control istransferred to step S11 of FIG. 26. But if no entry to or exit from anytunnel is included in the image acquired by the camera 3, then thecontrol device 4 reaches a negative decision in step S110, and the flowof control proceeds to step S120. It should be understood that if,although an affirmative decision was reached in step S110 when makingthis decision during the previous iteration of this routine, this haschanged to a negative decision in step S110 this time, then the controldevice 4 cancels the setting of the frame rate that will be describedhereinafter on the basis of the flow chart of FIG. 26.

In step S111 of FIG. 26, from the image acquired by the camera 3, thecontrol device 4 detects the bright portion external to the tunnel 83and the dark portion internal to the tunnel 83, and then the flow ofcontrol proceeds to step S112. For example, the control device 4 maydetect bright portions and dark portions (shown by sloping lines) fromthe image of the photographic subject shown by way of example in FIG.27.

Then in step S112 of FIG. 26 the control device 4 takes the region ofthe dark portion in the first imaging region 71 as being a third imagingregion 71 a, and takes the region in the first imaging region 71 otherthan the third imaging region 71 a as being remainder region 71 b asshown in FIG. 27. In other words, the first imaging region 71 is dividedinto the third imaging region 71 a and the remainder region 71 b.

In step S113, the control device 4 sends a command to the camera 3, andsets the frame rate in the third imaging region 71 a to be lower thanthe frame rate in the remainder region 71 b. For example, the controldevice 4 may reduce the frame rate in the third imaging region 71 a to60 fps. This is done in order to obtain clear image information from thedark third imaging region 71 a which represents the interior of thetunnel 83. On the other hand, the frame rate of the remainder region 71b is kept just as it is at 120 fps.

In step S114, the control device 4 further takes the dark portion in thesecond imaging region 72 as being a fourth imaging region 72 a, andtakes the region in the second imaging region 72 other than the fourthimaging region 72 a as being a remainder region 71 b as shown in FIG.27. In other words, the second imaging region 72 is divided into thefourth imaging region 72 a and the remainder region 72 b.

In step S115, the control device 4 sends a command to the camera 3, andsets the frame rate in the fourth imaging region 72 a to be lower thanthe frame rate in the remainder region 72 b. For example, the controldevice 4 may reduce the frame rate in the fourth imaging region 72 a to30 fps. This is done in order to obtain clear image information from thedark fourth imaging region 72 a which represents the interior of thetunnel 83. And then the control device 4 advances the flow of control tostep S40 (of FIG. 24).

It should be understood that while, in the explanation given above, thecase of a tunnel entry was explained, the same holds for the case of atunnel exit. However, in the ease of a tunnel entry and in the case of atunnel exit, the relationship between the bright portions of the imageand its dark portions is opposite: the image of the exterior of thetunnel as seen from the interior of the tunnel becomes the brightportion.

FIG. 28 is a flow chart showing an example of processing upon change toa second traveling environment (i.e. to a second road state). And FIG.29 consists of two figures schematically showing images of aphotographic subject that are focused upon the imaging surface of theimage capture chip 113 when the headlights of the vehicle 1 are turnedon: FIG. 29(a) is a figure showing the case of high beam, and FIG. 29(b)is a figure showing the case of low beam.

In step 120 of FIG. 25, the control device 4 makes a decision as towhether or not the vehicle headlights are turned on. If a bright portiondue to the headlights is included in the image acquired by the camera 3,then the control device 4 reaches an affirmative decision in step S120and the flow of control is transferred to step S121 of FIG. 28. But ifno bright portion due to the headlights is included in the imageacquired by the camera 3, then the control device 4 reaches a negativedecision in step S120 and the flow of control proceeds to step S130. Andif, although an affirmative decision was reached in step S120 whenmaking this decision during the previous iteration of this routine, thischanges to a negative decision in step S120 this time, then the controldevice 4 cancels the setting of the frame rate that will be describedhereinafter on the basis of the flow chart of FIG. 28.

It should be understood that, instead of determining that the headlightsare turned on based upon detection of a bright portion in the image, itwould also be acceptable to arrange to determine that the headlights areturned on based upon turning on operation by the driver. In this case,if the beam changeover switch 18 is changed over to the high beam side,then the control device 4 treats the position of the illuminated region86 a in FIG. 29(a) as a bright portion of the image acquired by thecamera 3. Moreover, if the beam change over switch 18 is changed over tothe low beam side, then the control device 4 treats the position of theilluminated region 86 b in FIG. 29(b) as a bright portion of the imageacquired by the camera 3.

In step S121 of FIG. 28, the control device 4 detects the regionilluminated by the headlights (corresponding to the bright portiondescribed above) from the image acquired by the camera 3, and then theflow of control proceeds to step S122.

In step S122, if the beam changeover switch 18 is changed over to thehigh beam side, as shown in FIG. 29(a), the control device 4 takes theregion of overlapping of the first imaging region 73 and the illuminatedregion 86 a as being a fifth imaging region 73 a, and takes a regionconsisting of the first imaging region 73 with this fifth imaging region73 a eliminated as being a remainder region 73 b. In other words, thefirst imaging region 73 is separated into the fifth imaging region 73 aand the remainder region 73 b.

In step S123, the control device 4 sends a command to the camera 3, andsets the frame rate in the fifth imaging region 73 a to be higher thanthe frame rate in the remainder region 73 b. For example, if the framerate in the remainder region 73 b is 60 fps, the control device 4 mayincrease the frame rate in the fifth imaging region 73 a to 120 fps.This is done in order to enhance the frequency of acquiring imageinformation from the fifth imaging region 73 a, which becomes brightwhen illuminated at high beam.

In step S124, as shown in FIG. 29(a), the control device 4 takes theregion of overlapping of the second imaging region 74 and theilluminated region 86 a as being a sixth imaging region 74 a, and takesa region consisting of the second imaging region 74 with this sixthimaging region 74 a eliminated as being a remainder region 74 b. Inother words, the second imaging region 74 is separated into the sixthimaging region 73 a and the remainder region 74 b.

In step S125, the control device 4 sends a command to the camera 3, andsets the frame rate in the sixth imaging region 74 a to be higher thanthe frame rate in the remainder region 74 b. For example, if the framerate in the remainder region 74 b is 60 fps, the control device 4 mayincrease the frame rate in the sixth imaging region 74 a to 120 fps.This is done in order to enhance the frequency of acquiring imageinformation from the sixth imaging region 74 a which becomes bright whenilluminated at high beam. And then the control device 4 advances theflow of control to step S40 (of FIG. 24).

While the situation during high beam has been explained with referenceto FIG. 29(a), it is possible to perform the same operation in a similarmanner during low beam as well. When the beam changeover switch 18 ischanged over to the low beam side, then the illuminated region 86 b inFIG. 29(b) corresponds to the bright portion of the photographic subjectimage. And, to correlate FIG. 29(a) with FIG. 29(b), the fifth imagingregion 73 a corresponds to a fifth imaging region 73 c, the remainderregion 73 b corresponds to a remainder region 73 d, the sixth imagingregion 74 a corresponds to a sixth imaging region 74 c, and theremainder region 74 b corresponds to a remainder region 74 d.

FIG. 30 is a flow chart showing an example of processing upon change toa third traveling environment (i.e. to a third road state). And FIG. 31is a figure schematically showing an image of a photographic subjectthat is focused upon the imaging surface of the image capture chip 113when the tail lamps (tail lights) of the leading vehicle 84 in front areturned on.

In step 130 of FIG. 25, the control device 4 makes a decision as towhether or not a tail lamp of the leading vehicle 84 in front has beenrecognized. If a tail lamp in an illuminated state has been recognizedfrom the image acquired by the camera 3, then the control device 4reaches an affirmative decision in step S130 and the flow of control istransferred to step S131 of FIG. 30. But if no tail lamp in anilluminated state has been recognized from the image acquired by thecamera 3, then the control device 4 reaches a negative decision in stepS130 and the flow of control proceeds to step S140. It should beunderstood that if, although an affirmative decision was reached in stepS130 when making this decision during the previous iteration of thisroutine, this changes to a negative decision in step S130 this time,then the control device 4 cancels the setting of the frame rate thatwill be described hereinafter on the basis of the flow chart of FIG. 30.

In step S131 of FIG. 30, the control device 4 detects the tail lamp ofthe leading vehicle 84 from the image acquired by the camera 3, and thenthe flow of control proceeds to step S132. In FIG. 31, an image of thetail lamp 84 a is not included in the first imaging region 75 of FIG.31, but is included in the second imaging region 76.

In step S132 of FIG. 30, the control device 4 takes a predeterminedregion that includes an image of the tail lamp 84 a of the leadingvehicle 84, for example, a rectangular region including the tail lamps84 a on both sides, as being a seventh imaging region 87. And in stepS133, in order to perform charge accumulation control for this seventhimaging region 87 under the same conditions as those for the firstimaging region 75, the control device 4 separates the seventh imagingregion 87 from the second imaging region 76, and incorporates it intothe existing first imaging region 75. It should be understood that theshape of this seventh imaging region 87 is not limited to beingrectangular; it would also be acceptable for it to be an ellipse or atrapezoid that includes the tail lamps 84 a on both sides.

The control device 4 further issues a command to the camera 3, so as toset the frame rate for the seventh imaging region 87 to the same framerate as that of the first imaging region 75. For example, if the framerate for the first imaging region 75 is 120 fps, then the frame rate forthe seventh imaging region 87 is also raised to 120 fps. This is inorder to enhance the frequency of acquiring image information for theseventh imaging region 87, which corresponds to the leading vehicle 84.And then the control device 4 transfers the flow of control to step S40(of FIG. 24).

FIG. 32 is a flow chart showing an example of processing upon change toa fourth traveling environment (i.e. to a fourth road state). And FIG.33 is a figure schematically showing an image of a photographic subjectthat is focused upon the imaging surface of the image capture chip 113when rainfall has been detected.

In step 140 of FIG. 25, the control device 4 makes a decision as to thepresence or absence of rainfall information from the rainfall sensor 19(refer to FIG. 1). If rainfall information is being inputted, then thecontrol device 4 reaches an affirmative decision in step S140 and theflow of control is transferred to step S141 of FIG. 32. But if norainfall information is being inputted, then the control device 4reaches a negative decision in step S140 and the flow of controlproceeds to step S150. It should be understood that if, although anaffirmative decision was reached in step S140 when making this decisionduring the previous iteration of this routine, this changes to anegative decision in step S140 this time, then the control device 4cancels the setting of the frame rate that will be described hereinafteron the basis of the flow chart of FIG. 32.

Generally, during rainfall, it becomes difficult to identify the whitelines drawn upon the road, since the road surface is wet. In concreteterms, as compared to a dry road surface, the contrast between the whiteline portions and the portions other than white lines decreases.

In step S141 of FIG. 32, in a similar manner to the case in step S35 ofFIG. 25, the control device 4 separates a region of attention in theimage of the photographic subject that is focused upon the image capturechip 113, which is taken as being a first imaging region 77, from theregion other than this first imaging region 77, which is taken as beinga second imaging region 78, and then the flow of control proceeds tostep S142. To explain this with reference to FIG. 33, the control device4 takes the trapezoidally shaped region that includes the white line 82a, the white line 82 b, and the region between the white line 82 a andthe White line 82 b as being a first imaging region 77, and takes theregion other than this first imaging region 77 as being a second imagingregion 78.

in step S142 of FIG. 32, the control device sends a command to thecamera 3, and sets the frame rate for the first imaging region 77 to belower than the frame rate in the case of the first imaging region 71(refer to FIG. 23). For example, if the frame rate for the first imagingregion 71 is 120 fps, then the frame rate for the first imaging region77 may be changed to 60 fps. This is done in order to obtain clear imageinformation from the first imaging region 77, the luminance of whosewhite colored portions has decreased because the road surface is wet.And then the control device transfers the flow of control to step S150of FIG. 25,

It should be understood that, instead of decreasing the frame rate inthis manner, it would also be acceptable to increase the contrast of theimage by adjusting the tone curve.

Furthermore, for a similar reason, it would also be acceptable to setthe frame rate for the second imaging region 78 to be lower than theframe rate in the case of the second imaging region 72 (refer to FIG.23); and it would also be acceptable to increase the contrast of theimage by adjusting the tone curve.

FIG. 34 is a flow chart showing an example of processing upon change toa fifth traveling environment (i.e. to a fifth road state). And FIG. 35is a figure schematically showing an image of a photographic subjectthat is focused upon the imaging surface of the image capture chip 113when a change of course has been detected. FIG. 35(a) shows the image ofthe photographic subject before the change of vehicle lane, while FIG.35(b) shows the image of the photographic subject during the change ofvehicle lane.

In step 150 of FIG. 25, the control device 4 makes a decision as to thepresence or absence of information related to a change of vehicle lane.By comparing and matching position information inputted from the GPSdevice 15 with map information, a navigation device not shown in thefigures performs, for example, route guidance as to which road thevehicle 1 is to travel upon (i.e. In which lane the vehicle shouldtravel) and in what azimuth the vehicle should travel. If a change ofvehicle lane by the vehicle 1 is required, then a vehicle lane changecommand is inputted from the navigation device described above to thecontrol device 4. If such a vehicle lane change command has beeninputted, then the control device 4 reaches an affirmative decision stepS150 and the flow of control is transferred to step S151 of FIG. 34. Butif no such vehicle lane change command has been inputted, then thecontrol device 4 reaches a negative decision in step S150 and the flowof control is transferred to step S40 of FIG. 24.

It should be understood that if, although an affirmative decision wasreached in step S150 when making this decision during the previousiteration of this routine, this changes to a negative decision in stepS150 this time, then the control device 4 cancels the setting of theframe rate that will be described hereinafter on the basis of the flowchart of FIG. 34.

In step S151 of FIG. 34, the control device 4 makes a change to thefirst imaging region 79 in the image of the photographic subject that isfocused upon the image capture chip 113 in the following manner.

To explain with reference to FIG. 35(a), in a similar manner to the casein step S33, the control device 4 takes the trapezoidally shaped regionthat includes the white line 87 a, the white line 87 b, and the regionbetween the white line 87 a and the white line 87 b as being a firstimaging region 79, and takes the region other than this first imagingregion 79 as being a second imaging region 80.

If the position information inputted from the GPS device 15 correspondsto a changed position of vehicle lane, then the control device 4 changesthe first imaging region 79 to a first imaging region 79A, as describedbelow. To explain with reference to FIG. 35(b), since the new region ofattention becomes the white line 87 b that delimits the traffic laneafter change of vehicle lane and the white line 87 c, accordingly thefirst imaging region 79A is a trapezoidal shape that includes the whitelines 87 b and 87 e. Thus, the control device 4 gradually displaces theimaging region from the first imaging region 79 shown in FIG. 35(a) tothe first imaging region 79A shown in FIG. 35(b). The processing toshift the imaging region in this manner is performed continuouslyaccording to change of the photographic field along with the change ofvehicle lane.

It should be understood that, instead of operating to shift the imagingregion, it would also be acceptable to increase the size of the firstimaging region 79 so that it includes the first imaging region 79A.

In step S152 of FIG. 34, the control device 4 sends a command to thecamera 3, and sets the frame rate of the first imaging region 79A thathas been reset to the same frame rate as that of the first imagingregion 79. Due to this, when performing change of course, it is possibleto keep the frame rate of the region of attention after the change ofvehicle lane to be the same as the frame rate of the region of attentionbefore the change of vehicle lane. And then the control device transfersthe flow of control to step S40 (refer to FIG. 24).

According to this second embodiment, the following beneficialoperational effects are obtained.

(1) The control device 4 sets to the camera 3 the first imaging region71 that includes the white lines 82 a and 82 b, and the second imagingregion 72 that is the portion other than the first imaging region 71,and sets the photographic conditions for the first imaging region 71 andthe photographic conditions for the second imaging region 72 to bedifferent from one another. Since, for example, the frame rate for thefirst imaging region 71 is set to be higher than the frame rate for thesecond imaging region 72, accordingly it is possible reliably torecognize the white lines 82 a and 82 b in the image acquired by thecamera 3. Furthermore, since the frame rate for the second imagingregion 72 that does not contribute to recognition of the white lines 82a and 82 b is set to be lower, accordingly it is possible to reduce theconsumption of electrical power by the camera 3, and it is possible tosuppress the generation of heat.

When the frame rate is high the charge accumulation time becomes short,and when the frame rate is low the charge accumulation time becomeslong.

(2) Since, even when the traveling environment of the vehicle 1 haschanged, according to the brightness or darkness of the first imagingregion 71, the control device 4 divides the first imaging region 71 intotwo imaging regions, and makes the frame rates of these two imagingregions be different, accordingly it is possible to continue reliably torecognize the white lines 82 a and 82 b in the images acquired by thecamera 3.

For example since, in the case of entry to a tunnel 82 as shown in FIG.27, along with taking the dark portion in the first imaging region 71 asbeing the third imaging region 71 a and taking the other region as beingthe remainder region 71 b, also the frame rate in this third imagingregion 71 a is set to be lower than the frame rate in the remainderregion 71 b, accordingly it is possible clearly to recognize the whitelines 82 a and 82 b within the third imaging region 71 a in the imageacquired by the camera 3. And, by performing similar control for thesecond imaging region 72 as for the first imaging region 71, it ispossible clearly to recognize the imaging region that is related totraveling in the image acquired by the camera 3.

Furthermore since, during high beam illumination as shown for example inFIG. 29(a), along with the region in which the first imaging region 73and the illuminated region 86 a overlap being taken as being the fifthimaging region 73 a and the other region being taken as being theremainder region 73 b, also the frame rate of the fifth imaging region73 a is set to be higher than the frame rate of the remainder region 73b, accordingly it is possible reliably to recognize the white lines 82 aand 82 b within the fifth imaging region 73 a in the image acquired bythe camera 3. And by performing similar control for the second imagingregion 72 as that performed for the first imaging region 71, it ispossible to increase the amount of information in the image acquired bythe camera 3 for the imaging region that is related to traveling.

(3) If, as an example of change of the traveling environment of thevehicle 1, the tail lamps of a leading vehicle 84 in front have beenrecognized (refer to FIG. 31), then, by taking a region that includesthe tail lamps 84 a on both sides as being a seventh imaging region 87,and by setting the frame rate for this seventh imaging region 87 to beequal to the frame rate for the first imaging region 75, it is possiblereliably to recognize the tail lamps 84 a of the leading vehicle 84 inthe image acquired by the camera 3.

(4) If, as an example of change of the traveling environment of thevehicle 1, rainfall has been detected (refer to FIG. 33), then, sincethe frame rate of the first imaging region 77 is set to be lower thanbefore the rainfall, accordingly it is possible easily to recognize thewhite lines 82 a and 82 b, whose contrast on the wet road surface hasdropped, in the image acquired by the camera 3.

(5) And if, as an example of change of the traveling environment of thevehicle 1, a change of vehicle lane is performed (refer to FIG. 35),then, since the frame rate of the first imaging region 79A that includesthe white lines 87 b after the change of vehicle lane is set to be thesame as the frame rate of the first imaging region 79 before the changeof vehicle lane, accordingly the frame rate of the first imaging regionis kept constant before and after the change of vehicle lane, so that itis possible reliably to recognize the white lines in the images acquiredby the camera 3 both before and after the change of vehicle lane.

One or a plurality of variant embodiments such as the following can alsobe combined with the second embodiment described above.

Variant Embodiment #1

While, in this second embodiment, an example of control of the camera 3under control of the control device 4 was explained, it would also beacceptable to provide a structure in which part of the control of thecamera 3 is performed by the control unit 35 of the camera 3.

Moreover while, in this second embodiment, among the photographicconditions, principally the frame rate has been described, it would alsobe acceptable to vary one or more photographic conditions other than theframe rate between the various photographic (image capture) regions.

Variant Embodiment #2

While, in the second embodiment, an example was explained in which thewhite lines drawn along the surface of the road were recognized as beingobjects to which attention should be directed, it would also beacceptable to include, not white lines, but rather guard rails or curbsor the like that are provided along the road as being objects to whichattention should be directed.

Variant Embodiment #3

In the second embodiment, an example was explained in which the imagingsurface 70 was divided into two portions, i.e. a first imaging region oftrapezoidal shape and a second imaging region other than the firstimaging region, and the frame rates for these two regions were set to bedifferent. Instead of the above, it would also be acceptable to dividethe imaging surface 70 into three or more imaging regions, i.e. a firstimaging region, a second imaging region, and a third imaging region, andto set the frame rates to be different for each of these imagingregions.

Moreover, in the second embodiment, an example was explained in whicheach of the first imaging region and the second imaging region wasfurther divided into two imaging regions. Instead of the above, it wouldalso be acceptable to arrange to divide each of the above describedfirst imaging region and the above described second imaging region intothree or more imaging regions, and to set the frame rates to bedifferent for each of these three or more finely divided imagingregions.

Variant Embodiment #4

In the second embodiment it was arranged for the control device 4, uponthe trigger of a vehicle lane change command being inputted from thenavigation device not shown in the figures, to shift the imaging regiongradually from the first imaging region 79 shown in FIG. 35(a) to thefirst imaging region 79A shown in FIG. 35(b). Instead of the above, itwould also be acceptable to arrange for the timing at which the changeof vehicle lane is performed to be determined by the control device 4.The control device 4 receives supply of information about ground pointswhere change of vehicle lane should be performed from a navigationdevice of the vehicle, and stores this ground point information in thestorage device 4 b in advance. And the control device 4 determines thatthe vehicle lane is to be changed when the position information inputtedfrom the GPS device 15 agrees with the information about a ground pointwhere change of vehicle lane should be performed. Moreover, havingdetermined upon the change of vehicle lane, when the control device 4detects the vehicle lane after the change of vehicle lane in the imageacquired by the camera 3, it shifts the imaging region from the firstimaging region 79 shown in FIG. 35(a) to the first imaging region 79Ashown in FIG. 35(b).

Variant Embodiment #5

While, in the second embodiment, as a method of distance measurementperformed by the camera 3, a calculation technique was employed in whichrange finding calculation was performed by using the image signals fromthe pixels for focus detection that were provided to the imaging element100, it would also be acceptable to employ a technique of performingdistance measurement by using two images from a stereo camera. Moreover,it would also be possible to employ a technique for performing distancemeasurement by using a millimeter wave radar separate from the camera 3.

Embodiment #3

In this embodiment, charge accumulation control for an imaging element100 (i.e. an image capture chip 113 (refer to FIG. 3)) of aphotoelectric conversion unit 15 of an image capture unit 5 (refer toFIG. 36) is performed by the block units described above, on the basisof the method of driving of another vehicle (for example anautomatically driven vehicle or a manually driven vehicle) that istraveling in the vicinity of the vehicle 1.

FIG. 36 is a block diagram showing an example of the structure of animage capture system 1 that includes an image capture device accordingto this third embodiment. This image capture system 1 employs anautomobile 10, other vehicle 20, a traffic signal generation device 30,and a traffic light 40.

It should be understood that, instead of the traffic light 40, or inparallel with the traffic light 40, it would also be acceptable toemploy an information supply system that is installed to the road, or aVICS (registered trademark; Vehicle Information and CommunicationSystem).

While there are some reference symbols in FIGS. 36, 40, and 42 through45 that are common to reference symbols used in previous drawings, herethey are applied to the explanation of this third embodiment.

The Automobile 10

The automobile 10 comprises a vehicle actuation section 11, a GPS device12, a navigation system 13, an optical system 14, a photoelectricconversion unit 15, a communication unit 16, a storage unit 17, a sensor18, and a control unit 19.

It should be understood that the portions of the automobile 10 that arenot explained herein have the same basic structures as in a conventionalautomobile.

The vehicle actuation section 11 includes actuation members of varioustypes related to operation of the automobile, such as a steering wheel,a turn signal switch, a shift lever, an accelerator, a brake, a switchfor changing over between automatic driving mode and manual drivingmode, and so on.

On the basis of signals obtained by reception of radio waves from theGPS satellites, the GPS device 12 calculates the position of theautomobile 10 (i.e. its longitude, latitude, and so on). And theposition information that has thus been calculated by the GPS device 12is outputted to the navigation system 13 and/or to the control unit 19.

The navigation system 13 is a system that provides guidance along atravel path to a destination that has been inputted by detecting thecurrent position of the automobile 10 from the UPS device 12 or thelike, by acquiring map data corresponding to this current position froma storage medium or a network, and by displaying this map data upon aliquid crystal monitor. This navigation system 13 comprises an actuationunit that receives actuation from a user, the liquid crystal monitormentioned above, a speaker that provides audio guidance, a reading unitthat reads map data, and so on.

The optical system 14 is built from a plurality of lenses, and focusesan image of the photographic subject upon the photoelectric conversionunit 15. If the optical system 14 is facing forward from the automobile10, then an image forward in the direction of travel of the automobile10 is acquired by the photoelectric conversion unit 15. And, if theoptical system 14 is facing rearward from the automobile 10, then animage rearward in the opposite direction to the direction of travel ofthe automobile 10 is acquired by the photoelectric conversion unit 15.The optical system 14 has an angle of view that is adapted to aplurality of lanes of travel (two vehicle lanes or three vehicle lanesor the like).

It should be understood that it would also be acceptable to provide astereo camera that has a plurality of optical systems 14.

The photoelectric conversion unit 15 comprises an imaging element 100that comprises, laminated together, an image capture chip that outputspixel signals corresponding to light incident from the optical system14, a signal processing chip that processes the pixel signals, and amemory chip that stores the pixel signals. As will be described indetail hereinafter, this imaging element 100 is capable of settingindividually different image capture conditions for each pixel or foreach of a number of unit regions each consisting of a plurality ofpixels (for example 16 pixels by 16 pixels), also including the casethat image capture is not to be performed thereby.

In this embodiment, the image capture unit 5 of the camera comprises theoptical system 14 and the photoelectric conversion unit 15, and capturesimages of objects in the vicinity of the automobile 10 (i.e. movingbodies and obstructions and so on), and also captures images of whitelines upon the road (also including lines of other colors such as yellowor the like). A forward image capture unit 5 that acquires imagesforward from the automobile 10 and a rearward image capture unit 5 thatacquires images rearward from the automobile 10 are provided to theautomobile 10.

In this explanation, lines that are white in color or the like and thatextend along the path of travel are termed “white lines”. Moreover, bothsolid lines and broken lines are termed “white lines”.

It should be understood that it would also be acceptable to provide aradar, although this is not shown in the figures, and to arrange todetect objects in the vicinity with this radar and the image captureunit 5 (i.e. the optical system 14 and the photoelectric conversion unit15).

The communication unit 16 performs wireless communication (including anoptical beacon, a radio beacon, visible light communication, and thelike) with external devices, such as other vehicles 20 and trafficlights 40 and so on. Any desired type of communication method may beemployed.

The storage unit 17 may, for example, be built as a nonvolatilesemiconductor memory such as a flash memory or the like, and storesprograms and control parameters of various types for travel of theautomobile 10 (including automatic travel).

The sensor 18 includes one or a plurality of sensors of various types,such as a vehicle speed sensor, a yaw rate sensor, or the like. Thevehicle speed sensor detects the speed V of the automobile 10, and sendsits detection signal to the control unit 19 or the like. And the yawrate sensor detects the yaw rate of the automobile 10, and sends itsdetection signal to the control unit 19 or the like. The yaw rate is therate of change of the rotational angle of the vehicle or the like in theyaw direction.

The control unit 19 is a device that controls the automobile 10 as awhole, and comprises a CPU (Central Processing Unit), RAM (Random AccessMemory), ROM (Read Only Memory) and so on. In this embodiment, thecontrol unit 19 performs setting of the image capture conditions foreach unit region of the imaging element 100 of the photoelectricconversion unit 15 and control thereof. Moreover, if the automaticdriving mode is set by the vehicle actuation section 11, then thecontrol unit 19 detects white lines upon the road with the image captureunit 5 (i.e. the optical system 14 and the photoelectric conversion unit15) and also detects mobile objects and obstructions in the vicinity ofthe automobile 10 by using the image capture unit 5, and, in cooperationwith the navigation system 13, performs automatic driving to adestination that has been inputted to the navigation system 13.

It should be understood that, in this embodiment, the automatic drivingmode means that operation and so on of the steering wheel, of theaccelerator, of the brake, of the turn signal switch, and of the shiftlever are all automatically performed under control by the control unit19. Moreover, the manual driving mode means that actuation and so on ofthe steering wheel, of the accelerator, of the brake, of the turn signalswitch, and of the shift lever are performed by the driver; in somecases the speed change mechanism is an automatic transmission, and insome cases it is a manual transmission. Furthermore, in addition tofully automatic driving in which all of the driving functions areperformed under the control of the control unit 19, the automaticdriving mode also includes semi-automatic driving in which, even thoughthe user is performing actuation of the vehicle actuation unit 11, thecontrol unit 19 may stop or decelerate the automobile 10 so as to avoidcollision and the like on the basis of the outputs from the imagecapture unit 5, the GPS 12, the communication unit 16, the sensor 18,and so on. Due to this, it is possible to ensure safety while the userstill can enjoy driving the automobile 10. Moreover, semi-automaticdriving also includes the case in which the control unit controls somebut not all of the steering wheel, the accelerator, the brake, the turnsignal switch, and/or the shift lever, instead of the driver.

The Other Vehicle 20

The other vehicle 20 comprises a communication unit 21, a vehicleactuation section 22, a storage unit 23, an image capture unit 24, acontrol unit 25 and so on, and the functions of these various sectionsare the same as functions of the corresponding sections of theautomobile 10. While various sections of the other vehicle 20 areomitted in FIG. 36, they support its basic structure as an automobile.However, the other vehicle 20 may be a vehicle of a type that is notequipped with any communication unit 21. Moreover, automatically drivenvehicles and manually driven vehicles may be mixed together as othervehicles 20. Of these vehicles, at least those vehicles that areequipped with an automatic driving mode are capable of communicatingwith one another via their communication units 21, and are built so asto be able to transmit and receive information related to whether theyare being automatically driven or are being manually driven, and also soas to be able to communicate information related to image data acquiredby their image capture units 24.

The Traffic Signal Generation Device 30

The traffic signal generation device 30 is a device for performingcontrol of signal lights that are displayed upon the display unit 42 ofthe traffic light 40, and comprises a signal information generation unit31, a storage unit 32, a communication unit 33, and a control unit 34.While such a traffic signal generation device 30 may be installed toeach of a plurality of traffic lights 40 that are provided at variousintersections, it would also be acceptable to arrange for a singletraffic signal generation device 30 to control a plurality of trafficlights 40.

The signal information generation unit 31 generates traffic signals onthe basis of the types and the positions of installation of a pluralityof traffic lights 40 that are installed at intersections or the like,and on the basis of commands or the like related to traffic sent from atraffic management center not shown in the figures.

The storage unit 32 may be built as a non-volatile semiconductor memorysuch as, for example, a flash memory or the like, and stores programs ofvarious types and control parameters and so on for the traffic signalgeneration device 30.

The communication unit 33 transmits traffic signals generated by thesignal information generation unit 31 to each of one or a plurality oftraffic lights 40 by cable or by wireless. Moreover, the communicationunit 33 may perform transmission and receipt of information to and froma traffic management center.

And the control unit 34 is a device that controls the traffic signalgeneration device 30 as a whole, and includes a CPU, RAM, ROM, and soon. Moreover, the control unit 34 may perform analysis related to thestate of traffic on the basis of the amount of traffic and so on, andmay perform control of the signal information generation unit 31 on thebasis thereof.

The Traffic Light 40

The traffic light 40 comprises a communication unit 41, a display unit42, an optical system 43, a photoelectric conversion unit 44, a storageunit 45, and a control unit 46. While only one such traffic light 40 isshown in FIG. 36, normally a plurality of traffic lights 40 areprovided. For example, in the case of an intersection, as shown by wayof example in FIG. 37, a total of eight traffic lights are provided:four traffic lights for automobiles 40 a, and four traffic lights forpedestrians 40 a. The traffic lights 40 receive traffic signals comingfrom the traffic signal generation device 30 according to the respectivepositions in which they are installed, and illuminate or blink displaylights of their display units 42.

The communication unit 41 receives traffic signals generated by thesignal information generation unit 31 via cable or via wireless.Moreover, the communication unit 41 transmits and receives informationof various kinds, such as information related to driving of vehicles andinformation related to traffic and so on, to and from the automobile 10,other vehicles 20, and other traffic lights 40.

The display unit 42 incorporates signal lights, and performs display ofthose signal lights according to receipt of traffic signals by thecommunication unit 41. In concrete terms, the display unit 42illuminates, blinks, or extinguishes the signal lights in order topermit or restrict movement such as travel, stopping and so on ofvehicles traveling upon the road or of pedestrians crossing the road. Itshould be understood that it would be acceptable for the display unit 42not only to illuminate a red colored light, a yellow colored light, or agreen colored light, but also to illuminate arrow lights that indicatewhether, at the intersection, it is possible to go straight on, to turnleft, or to turn right.

In this embodiment, the image capture unit 50 of the camera comprisesthe optical system. 43 and the photoelectric conversion unit 44. Theoptical system 43 includes a plurality of lenses, and focuses an imageof the photographic subject upon the photoelectric conversion unit 44.When the image capture unit 50 is provided to a traffic light forautomobiles 40 a, the optical system 43 is principally used foracquiring images of vehicles. But when the image capture unit 50 isprovided to a traffic light for pedestrians 40 a, the optical system 43is principally used for acquiring images of pedestrians (includingbicycles). The image capture unit 50 is provided in the vicinity of thedisplay unit 42 (i.e. of its signal light).

The photoelectric conversion unit 44 comprises an imaging element 100that comprises, laminated together, an image capture chip that outputspixel signals corresponding to light that is incident from the opticalsystem 43, a signal processing chip that processes those pixel signals,and a memory chip that stores the pixel signals. This photoelectricconversion unit has a similar structure to that of the photoelectricconversion unit 15 of the automobile 10, as described above. The imagingelement 100 is capable of setting image capture conditions correspondingto the traffic signal for each pixel or for each of a number of unitregions each consisting of a plurality of pixels (for example 16 pixelsby 16 pixels).

FIG. 38 is a figure showing an example of a traffic light forautomobiles 40 a that is disposed at an intersection. In FIG. 38, theimage capture unit 50 is installed to the traffic light 40 a. As itsoptical system 43, this image capture unit 50 includes, for example, awide angle lens, and has an angle of view that includes four vehiclelanes on both sides of a plurality of travel lanes (i.e. two vehiclelanes on each side, or the like) of the intersection shown by way ofexample in FIG. 37.

The storage unit 45 is built as a non-volatile semiconductor memory suchas, for example, a flash memory, and stores image data acquired by thephotoelectric conversion unit 44 and the like.

The control unit 46 is a device for controlling the traffic light 40 asa whole, and comprises a CPU, RAM, ROM, and so on. In this embodimentthe control unit 46, along with performing display control of the signallights of the display unit 42 according to traffic signals, alsocontrols capture of images using the imaging element 100 of thephotoelectric conversion unit 44.

It should be understood that, if the range of photography by the imagecapture unit 50 of the traffic light for pedestrians 40 a is included inthe range of photography by the image capture unit 50 of the trafficlight for automobiles 40 a, then it would be acceptable to omit theimage capture unit 50 of the traffic light for pedestrians 40 a (i.e.its optical system 43 and its photoelectric conversion unit 44).

Control of the Automobile

In the following, the control executed by the control unit 19 of theautomobile 10 will be explained with reference to the flow chart of FIG.19. It should be understood that it will be supposed that this flowchart is started when the automobile 10 starts, for example uponstarting of the engine or upon starting of the driving system or thelike. A program for executing the processing of the FIG. 39 flow chartis stored in a storage medium such as a ROM or the like in the controlunit 19, or in the storage unit 17 of the automobile 10.

In step S1 the control unit 19 starts image capture by the imagingelement 100 of the photoelectric conversion unit 15. As described above,the image capture unit 5 of the automobile 10 (i.e. the optical system14 and the photoelectric conversion unit 15) acquires images both infront of the automobile 10 and behind it.

Then in step S2, via the communication unit 16, the control unit 19performs communication with other vehicles 20 (including, in addition tovehicles that are traveling in the same lane as the automobile 10, alsovehicles whose direction of progression is the same as that of theautomobile 10 but that are traveling in different travel lanes). In thisembodiment, as shown in FIG. 40(a), it is supposed that a leadingvehicle 73A in front that is traveling in the same lane as theautomobile 10 is an automatically driven vehicle, while a vehicle 72A infront that is traveling in a neighboring travel lane (and that isprogressing in the same direction) is a manually driven vehicle.Furthermore, it is supposed that a vehicle 73B behind that is travelingin the same lane as the automobile 10 is an automatically drivenvehicle, while a vehicle 72B behind that is traveling in an adjacenttravel lane (and that is progressing in the same direction) is amanually driven vehicle.

Whether a vehicle in the vicinity of the automobile 10 is anautomatically driven vehicle or is a manually driven vehicle isdetermined on the basis of the result of communication via thecommunication unit 16, such as per se known vehicle-to-vehiclecommunication or the like. Moreover it would be acceptable, if anidentification mark or the like is displayed upon the other vehicle 20,to arrange to pertain the above determination on the basis of the resultof image capture by the image capture unit 5 (i.e. the optical system 14and the photoelectric conversion unit 15). Such an identification markmay be a predetermined mark or code that is displayed upon the body ofthe vehicle, or could also be provided by displaying identificationinformation upon a display unit, not shown in the figures, that isprovided upon the root of the vehicle or the like.

It should be understood that, for other vehicle 20 for which it is notpossible to determine whether it is an automatically driven vehicle oris a manually driven vehicle on the basis of the result of communication(communication not being possible), or on the basis of image capture, itwill be supposed that the control unit 19 assesses that vehicle as beinga manually driven vehicle.

Then in step S3 the control unit 19 sets the image capture conditionsdifferently for each of the above described unit regions 131 (refer toFIG. 4) of the imaging element 100 of the photoelectric conversion unit15. FIGS. 40(b) and 40(c) are figures showing images of the photographicsubject that are focused upon the imaging element 100 by image capturerespectively forward of the automobile 10 and rearward thereof. Althoughactually inverted images are focused, for convenience of understanding,the images shown in these figures are erect images. The white line 80 ashows the demarcation line on the left side of the road facing thedirection of travel, the white line 80 a shows the boundary line of thelane of travel (i.e. of the lane of this vehicle), and the white line 80c shows the demarcation line on the right side of the road.

Since, as described above, the vehicles 72A and 72B that are travelingahead and behind the automobile 10 in the next lane are manually drivenvehicles, accordingly the control unit 19 takes the region in FIG. 40(b)that includes the vehicle 72A as a region of attention 71A. And thecontrol unit 19 sets the frame rate of the unit regions of the imagingelement 100 corresponding to the region of attention 71A to be higherthan the frame rate in the normal region (for example 60 fps), whilesetting the decimation ratio to be 0% to 20% lower than in the normalregion.

In a similar manner, the control unit 19 takes the region in FIG. 40(c)that includes the vehicle 72B as being a region of attention 71(B). Andthe control unit 19 sets the frame rate of the unit regions of theimaging element 100 corresponding to the region of attention 71B to behigher than the frame rate in the normal region (for example 60 fps),while setting the decimation ratio to be 0% to 20% lower than in thenormal region. Furthermore, the control unit 19 changes this decimationratio according to the speed of movement of the automobile 10, oraccording to the relative speed of movement of the automobile 10relative to the other vehicles 20. For example, the decimation ratio maybe changed to be lower along with the relative speed of movementbecoming faster.

It should be understood that, while FIG. 40(b) is an example in whichall of the regions except the region of attention 71A are normalregions, it would also be acceptable to arrange to take the region thatsurrounds the vehicle 73A, which is an automatically driven vehicle, asbeing a secondary region of attention 74A, and to take the regions otherthan the region of attention 71A and the secondary region of attention74A as being normal regions. Moreover, while FIG. 40(c) is an example inwhich all of the regions except the region of attention 71B are normalregions, it would also be acceptable to arrange to take the region thatsurrounds the vehicle 73B which is an automatically driven vehicle asbeing a secondary region of attention 74B, and to take the regions otherthan the region of attention 71B and the secondary region of attention74A as being normal regions. It should be understood that it would alsobe acceptable to arrange for the control unit 19 to set the imagecapture conditions for the secondary regions of attention 74A and 74B tobe different for fully automatic driving and for semi-automatic driving.In this case, the control unit 19 may set the frame rate of the imagingelement 100 during semi-automatic driving to be higher than the framerate of the imaging element 100 during fully automatic driving.Moreover, it would also be acceptable to arrange for the control unit 19to set these imaging regions to normal regions during fully automaticdriving.

For the imaging elements 100 that capture images in front of theautomobile 10 and behind it, the control unit 19 sets the frame rate ofthe unit regions of the imaging elements 100 that correspond to thesecondary regions of attention 74A and 74B to be lower than the framerate of the normal regions (for example 30 fps), and sets theirdecimation ratios to be 30% to 60% of that of the normal regions.

Furthermore, in addition to the regions of attention 71A and 71Bdescribed above, the control unit 19 may also take regions in whichwhite lines upon the road are included, as being regions of attention.And, for the imaging elements 100 that capture images in front of theautomobile 10 and behind it, the control unit 19 sets the frame rate ofthe unit regions of the imaging elements 100 that correspond to theregions of attention to be higher than the frame rate of the normalregions (for example 60 fps), and sets their decimation ratios to be 0%to 20% of that of the noinial regions.

By changing the image capture conditions for each of the unit regions ofthe imaging element 100 during image capture of a manually drivenvehicle and of an automatically driven vehicle in this manner, it ispossible to utilize the imaging element 100 efficiently, and to suppressthe consumption of electrical power and the generation of heat.

In step S4 of FIG. 39, the control unit 19 makes a decision as towhether or not it is possible to communicate with a traffic light 40 viathe communication unit 16, in other words as to whether or not thevehicle has come close to a traffic light 40 (i.e. an intersection) orthe like. If it is not possible to communicate with a traffic light 40(i.e. if the vehicle is outside any area of communication), then thecontrol unit 19 reaches a negative decision in step S4 and the flow ofcontrol is transferred to step S7. On the other hand, if communicationwith a traffic light 40 is possible (i.e. if the vehicle is within anarea of communication), then the control unit 19 reaches an affirmativedecision in step S4 and the flow of control proceeds to step S5.

In step S5, the control unit 19 receives information on the basis of animage acquired by the photoelectric conversion unit 44 of the trafficlight 40 (which may be a traffic light for automobiles 40 a or a trafficlight for pedestrians 40 a). For example, when the automobile 10 turnsleft (or when a vehicle turns right in a region where traffic drives onthe right side, such as the US or the like), the control unit 19 mayreceive information related to people from the traffic light forpedestrians 40 a. In this case, the presence or absence of a pedestrianis determined on the basis of an image that the control unit 46 of thetraffic light 40 has acquired via the photoelectric conversion unit 44,and the control unit 19 receives information related to the pedestriandetermined by the control unit 46.

It should be understood that it would also be acceptable to arrange forthe control unit 19 of the automobile 10 to receive image data acquiredby the photoelectric conversion unit 44 of the traffic light 40, and todetermine upon the presence or absence of a pedestrian on the basis ofthis image data that has been received.

If the automobile 10 is turning right (or, in a region in which vehiclesdrive on the right side, if it is turning left), then the control unit19 receives information from the traffic light for automobiles 40 a asto whether a vehicle straight ahead in the opposite vehicle lane is anautomatically driven vehicle or is a manually driven vehicle.Furthermore, the control unit 19 receives from the traffic light forautomobiles 40 a information related to changing over of the trafficsignal (for example, signal changeover information such as informationto the effect that, after a certain number of seconds, the signal willchange over from a green signal to a red signal).

Then in step S6, on the basis of the information entered in the step S5,the control unit 19 sets image capture conditions for the imagingelement 100 of the photoelectric conversion unit 15. If, for example,the automobile 10 is turning left at an intersection, then the controlunit 19 may increase the frame rate of the frame rate of the unitregions of the imaging element 100 that correspond to the left side ofthe photographic screen to be higher than the frame rate of the unitregions that correspond to the right side of the photographic screen,and may change the above described frame rate in correspondence with thespeed of the automobile 10 or with the speed of movement of a pedestrian(for example a speed of 4 km/h).

For example, if an automobile 10 that has been moving at a speed of 50km/h has decelerated to 10 km/h in order to turn left, then the framerate of the unit regions that correspond to the left side of thephotographic screen is reduced as compared to before the deceleration.Moreover, if the automobile 10 turns left, then the control unit 19 setsthe image capture conditions for the imaging element 100, according towhether a pedestrian who is crossing on a crosswalk is getting nearer tothe automobile 10 or is getting further away from the automobile 1. Inother words, the control unit increases the frame rate of the unitregions of the imaging element 100 that correspond to a person who isgetting nearer to the automobile 10 and also lowers their decimationratio, while lowering the frame rate of the unit regions of the imagingelement 100 that correspond to a pedestrian who is getting further awayfrom the automobile 10 (in particular, that correspond to a person whohas already finished crossing a pedestrian crossing through which theautomobile 10 is going to pass) and raises their decimation ratio.

Furthermore, if the automobile 10 is going to turn right at anintersection, and if the other vehicle 20 that is coming straight onfrom the opposite vehicle lane is a manually driven vehicle, then thecontrol unit 19 makes the frame rate of the unit regions correspondingto the right side of the photographic screen relatively higher than theframe rate of the unit regions corresponding to the left side of thephotographic screen, and also lowers their decimation ratio. Moreover,if a pedestrian who is crossing a crosswalk toward which the vehicle isturning right is getting nearer to the automobile 10, then the framerate of the unit regions of the imaging element 100 corresponding to thepedestrian who is getting nearer is made to be yet higher, and theirdecimation ratio is further lowered.

And it would also be acceptable to arrange for the control unit 19, ifthe automobile 10 is turning right or turning left, to forecast theimaging region to which attention should be directed (i.e. to forecastthe imaging region in which there is a possibility that a pedestrian orthe like may appear) and to change the setting of the image captureconditions, in correspondence to the state of actuation of the turnsignal switch and/or to the amount of actuation of the steering wheel.

Moreover, the control unit 19 receives information related to changingover of the traffic signal from the traffic light for automobiles 40 a,and changes over the image capture conditions of the imaging element 100on the basis of that information. For example, if the control unit 19receives from the traffic light for automobiles 40 a information relatedto changing over of the traffic signal to the effect that it will changeover from a green signal to a red signal after a certain number ofseconds, and the automobile accordingly decelerates, then the controlunit may perform control so that the frame rate of the imaging element100 that captures images forward of the vehicle becomes lower thanbefore deceleration, and so that the decimation ratio becomes higher. Onthe other hand, the control unit 19 may perform control so that theimage capture conditions of the imaging element 100 that captures imagesbehind the vehicle are maintained just as it is.

It should be understood that since, when the automobile decelerates, itmay be predicted that a vehicle following behind will approach theautomobile 10, accordingly it would also be acceptable to performcontrol so that the frame rate of the imaging element 100 that capturesimages rearward of the vehicle becomes higher than before deceleration,and so that the decimation ratio becomes lower.

It would also be acceptable for the control unit 19, when the speed ofthe automobile 10 changes, to forecast the change of speed according tothe actuation amount of the brake or the accelerator (i.e. the amount bywhich the corresponding pedal is stepped upon), and to change thesettings of the image capture conditions accordingly.

In step S7, the control unit 19 makes a decision as to whether or notthe engine (or the driving system) is ON. If the engine (or the drivingsystem) is ON, then the control unit 19 reaches an affirmative decisionin step S7 and the processing of step S2 and subsequently is repeated.But if the engine (or the driving system) is OFF, then the control unit19 reaches a negative decision in step S7, and the processing of thisflow chart terminates.

Traffic Light Control

Next, the control of the traffic light 40 performed by the control unit46 will be explained with reference to the flow chart of FIG. 41. Aprogram for executing the processing according to the flow chart of FIG.41 is stored in a storage medium in the control unit 46 such as a ROM orthe like, or in the storage unit 45.

In step S10, the control unit 46 makes a decision as to whether atraffic signal has been received from the traffic signal generationdevice 30. If a traffic signal has been received from the traffic signalgeneration device 30, then the control unit 46 reaches an affirmativedecision in step S10 and the flow of control proceeds to step S11. Butif no traffic signal has been received from the traffic signalgeneration device 30, then the control unit 46 reaches a negativedecision in step S10 and waits for receipt of a signal.

In step S11, the control unit 46 performs display control of the displayunit 42. For example, according to the traffic signal that has beenreceived from the traffic signal generation device 30, the control unit46 may perform control to change over the signal light display of thedisplay unit 42 from red to green.

Then in step S12 the control unit 46 performs communication with one ora plurality of vehicles or with other traffic lights 40. One or morevehicles with which the control unit 46 is in communication, theautomobile 10, and other vehicles 20 that are equipped withcommunication units 21 may be included.

It should be understood that it would also be acceptable to arrange forthe other vehicles that are the subjects of communication to be vehiclesthat are within a predetermined range from an intersection or from atraffic light 40, and it would also be acceptable for them to bevehicles that are capable of communicating via an information supplysystem that is installed in the road but that is not shown in thefigures, or the like.

The control unit 46 acquires information from a vehicle that is thesubject of communication that specifies its method of operation, i.e.whether that vehicle or a vehicle in the vicinity of that vehicle is anautomatically driven vehicle or is a manually driven vehicle or thelike. Moreover, the control unit 46 acquires information from thevehicle that is the subject of communication related to the drivingstate of that vehicle or of a vehicle in the vicinity of that vehicle.For example, the control unit 19 or the control unit 25 of the vehiclethat is the subject of communication may predict a change of course atan intersection (i.e. a right turn or a left turn) from the state of aturn signal switch for operating winkers (i.e. direction indicators).The control unit 46 acquires, from the vehicle that is the subject ofcommunication, this information predicting a right turn or a left turnthat has been forecast by the vehicle that is the subject ofcommunication as the above described information related to the state ofdriving.

It should be understood that it would also be acceptable to arrange forthe control unit 46 to determine whether the vehicle that is the subjectof communication is an automatically driven vehicle or is a manuallydriven vehicle on the basis of the result of communication with thevehicle. Moreover, if an identification mark or the like is displayedupon the vehicle, then it would also be acceptable to arrange for thecontrol unit 46 to make this determination on the basis of the result ofimage capture by the image capture unit (i.e. by the optical system 43and the photoelectric conversion unit 44). Furthermore, it will also beacceptable to arrange for the control unit 46 to determine upon a changeof course by the vehicle (i.e. a right turn or a left turn) from theoperational state of the winkers of the vehicle based upon the result ofimage capture by the image capture unit (i.e. by the optical system 43and the photoelectric conversion unit 44); or it would be possible tomake the above determination according to whether the vehicle is in aleft turn lane or is in a right turn lane.

Yet further, the control unit 46 may also perform communication withanother traffic light 40, including both a traffic light for automobiles40 a or a traffic light for pedestrians 40 a, and may thus acquireinformation related to the state of traffic at an intersection or thelike. Even further, according to requirements, the control unit 46 mayperform communication with a traffic signal generation device 30 that isrelated to generation of traffic signals for traffic lights, and mayacquire information related to the state of display of traffic signalsin this manner.

In step S13, the control unit 46 sets image capture conditions for theimaging element 100 of the photoelectric conversion unit 44 on the basisof the state of display of the signal lights of the display unit 42, andon the basis of the information that was acquired in step S12. Thedetails of this setting of the image capture conditions for the imagingelement 100 of the photoelectric conversion unit 44 will be describedhereinafter.

Then in step S14 the control unit 46 performs image capture with theimage capture unit (i.e. with the optical system 43 and thephotoelectric conversion unit 44) under the image capture conditionsthat were set in step S14.

And in step S15 the control unit 46 transmits the image data acquired instep S14 via the communication unit 41 to the automobile 10, to theother vehicle 20, or to another traffic light 40 or the like. Moreover,in a similar manner, on the basis of the image data described above, thecontrol unit 46 may also transmit information that has been extracted byperforming image processing or image analysis, for example informationabout the lane in which the vehicle is traveling as estimated from datarelated to the direction of the vehicle and to its speed and so on, andby identifying the operational state of the winkers and so on.

Furthermore, it would also be acceptable for the control unit 46, on thebasis of the information that has thus been analyzed, to recognize theestimated lane of travel of the vehicle that is the subject ofcommunication (the automobile 10 or the other vehicle 20) or an objectthat may become an obstacle (including a vehicle or a pedestrian), togenerate a message on the basis of the result of that recognition, andto transmit that message from the communication unit 41 to the vehiclethat is the subject of communication. The message may, for example, be“Two-wheeled vehicle coming from behind”, “Pedestrian crossing”,“Oncoming vehicle straight ahead”, or the like. The control unit 46executes the processing from step S10 through step S15 repeatedly.

The Setting of Image Capture Conditions for the Traffic Light

FIG. 42 is a figure showing an example of control of the image captureconditions for an imaging element 100 of an image capture unit 50-1 thatis installed integrally with a traffic light for automobiles 40 a, orthat is installed in the neighborhood of such a traffic light, when thistraffic light for automobiles 40 a is displaying a green signal.

In FIG. 42, there are shown a traffic light for automobiles 40 a for alane of travel (vehicle lane) A, an image capture unit 50-1, and atraffic signal generation device 30-1. Other traffic lights and so on atthis intersection are omitted from this figure. Within the range of theimaging region 70 of the image capture unit 50-1, a control unit 46 ofthis traffic light for automobiles 40 a provides, as a region ofattention 71, a range (the hatched range) in which the vehicles aremoving because of a green signal.

The control unit 46 performs control so as, for unit regions of theimaging element 100 that correspond to the region of attention 71, toincrease the frame rate to be higher, and to lower the decimation ratioto be lower, as compared to other unit regions outside the region ofattention 71. Moreover, when a vehicle 72 present in the region ofattention 71 is not an automatically driven vehicle but rather is amanually driven vehicle, then the control unit 46 provides the regionthat surrounds that vehicle 72 as being a special region of attention75. And the frame rate of the unit regions of the imaging element 100that correspond to this special region of attention 75 is further raisedto be even higher than that of the unit regions that correspond to theregion of attention 71. The control unit 46 also performs control tolower the decimation ratio of the unit regions of the imaging clement100 that correspond to the special region of attention 75 to be evenlower than the decimation ratio of the unit regions that correspond tothe region of attention 71.

Furthermore, with regard to regions within the region of attention 71that include a vehicle 76 and a vehicle 77 that have paused for turningright and left respectively, the control unit 46 is able to performimage capture at high resolution by adjusting the decimation ratio forboth those regions to be lower than the unit regions corresponding tothe region of attention 71.

FIG. 43 is a figure showing an example of control of the image captureconditions for the imaging element 100 of the image capture unit 50-1that is installed integrally with the traffic light for automobiles 40a, or that is installed in the neighborhood of such a traffic light,when this traffic light for automobiles 40 a is displaying a red signal.

In FIG. 43, there are shown the traffic light for automobiles 40 a for alane of travel (vehicle lane) A, an image capture unit 50-1, and atraffic signal generation device 30-1. Other traffic lights and so on atthis intersection are omitted from this figure. Within the range of theimaging region 70 of the image capture unit 50-1, the control unit 46 ofthis traffic light for automobiles 40 a sets, as a region of attention71, a crosswalk that a pedestrian 90 is entering and the vicinitythereof (the hatched range).

The control unit 46 performs control so as, for unit regions of theimaging element 100 that correspond to the region of attention 71, toincrease the frame rate to be higher, and to lower the decimation ratioto be lower, as compared to other unit regions outside the region ofattention 71. Moreover, when the pedestrian 90 has been recognized, thenthe control unit 46 sets a region within a predetermined range includingthat pedestrian 90 as being a special region of attention 75. And theframe rate of the unit regions of the imaging element 100 thatcorrespond to this special region of attention 75 is further raised tobe even higher than that of the unit regions that correspond to theregion of attention 71. The control unit 46 also performs control tolower the decimation ratio of the unit regions of the imaging element100 that correspond to the special region of attention 75 to be evenlower than the decimation ratio of the unit regions that correspond tothe region of attention 71.

In FIG. 44, there is shown an example of control of image captureconditions for an image capture unit 50-2 that is installed integrallywith a traffic light for pedestrians 40 a, or that is installed in theneighborhood of such a traffic light. In FIG. 44, there are shown thetraffic light for pedestrians 40 a, the image capture unit 50-2, and atraffic signal generation device 30-2. Other traffic lights and so on atthis intersection are omitted from this figure. Within the range of theimaging region 70 of the image capture unit 50-2, the control unit 46 ofthis traffic light for pedestrians 40 a provides, as a region ofattention 71, a crosswalk that a pedestrian 90 is entering and thevicinity thereof (the hatched range).

The control unit 46 performs control so as, for unit regions of theimaging element 100 that correspond to the region of attention 71, toincrease the frame rate to be higher, and to lower the decimation ratioto be lower, as compared to other unit regions outside the region ofattention 71. Moreover, when the pedestrian 90 has been recognized, thenthe control unit 46 sets a region within a predetermined range includingthat pedestrian 90 as being a special region of attention 75. And theframe rate of the unit regions of the imaging element 100 thatcorrespond to this special region of attention 75 is further raised tobe even higher than that of the unit regions that correspond to theregion of attention 71. The control unit 46 also performs control tolower the decimation ratio of the unit regions of the imaging element100 that correspond to the special region of attention 75 to be evenlower than the decimation ratio of the unit regions that correspond tothe region of attention 71.

FIG. 45(a) is a figure showing an example of a situation, an image ofwhich has been captured by the image capture unit 50-2 that is installedto the traffic light for pedestrians 40 a. And FIG. 45(h) is a figurefor explanation of setting of image capture conditions on the basis ofthe results of photographic subject recognition using image dataacquired by the image capture unit 50-2. In FIG. 45(a), the imagecapture unit 50-2 described above having the imaging element 100 isinstalled to the traffic light for pedestrians 40 a. Since, according tothe imaging element 100 according to this embodiment, it is possible tomeasure not only movement in the up and down direction and the left andright direction but also movement in the depth direction, accordingly itis possible to measure the speed Vo at which the pedestrian 90, who isthe photographic subject, is moving.

In FIG. 45(h), within the imaging region of the image capture unit 50-2,the control unit 46 sets a range that includes the pedestrian 90 who iswalking over the crosswalk as being the region of attention 71. And thecontrol unit 46 makes the image capture conditions for the unit regionsof the imaging element 100 that correspond to this region of attention71 be different from the unit regions outside the region of attention71. At this time, the control unit 46 varies the image captureconditions in dependence upon the speed Vo of the pedestrian 90.

For example, when the absolute value of the speed Vo of the pedestrian90 is high, the control unit 46 may perform control so as to increasethe frame rate for the unit regions of the imaging element 100 thatcorrespond to the region of attention 71 that includes the pedestrian 90to be higher than the frame rate for the unit regions outside the regionof attention 71, while lowering the decimation ratio of those unitregions.

Moreover, it would also be acceptable for the control unit 46 to changethe image capture conditions for the region of attention 71 on the basisof the positional relationship between the pedestrian 90 and a nearbyvehicle or building. For example, if a vehicle that is operating itswinker in order to turn right or to turn left at the intersection ispresent in the imaging region 70, then, since there is a possibilitythat this vehicle may enter into the crosswalk, accordingly the controlunit 46 sets, in the region of attention 71, the above described regionthat is close to the vehicle described above as being a special regionof attention. And the control unit 46 raises the frame rate for the unitregions of the imaging element 100 that correspond to this specialregion of attention to be yet higher than that for the unit regions thatcorrespond to the region of attention 71. Also, the control unit 46performs control to lower the decimation ratio for the unit regions ofthe imaging element 100 that correspond to the special region ofattention to be even lower than the decimation ratio for the unitregions that correspond to the region of attention 71.

Furthermore, in the region of attention 71, it would also be acceptablefor the control unit 46 to make the image capture conditions bedifferent between a plurality of pixels or regions 78P shown by slantinglines and a plurality of pixels or regions 78S not shown by slantinglines. In the example of FIG. 45(b), the image capture conditions aremade to be different between pixels or areas that are adjacent in theup, down, left, and right directions in a checked pattern, but this modeshould not be considered as being limitative.

Yet further, when an object such as another pedestrian or a bicycle orthe like within the range of the imaging region 70 has been recognized,it will be acceptable for the control unit 46 to add another regionincluding each of this plurality of objects to the region of attention71. And, in this plurality of regions of attention 71, it would also bepossible to make the image capture conditions to be different between aplurality of pixels or regions 78P each shown by slanting lines and aplurality of pixels or regions 78S each not shown by slanting lines.

FIG. 46 is a figure showing an example of a situation, an image of whichhas been captured by the image capture unit 50-1 that is installed tothe traffic light 40 a for automobiles at an intersection. In FIG. 46,the traffic light for automobiles 40 a comprises a display unit 42 athat has display lamps for indicating that it is possible to go straightahead, that it is possible to turn left, and that it is possible to turnright. An example will now be explained of image capture conditions thatare based upon the results of photographic subject recognition using theimage data captured by the image capture unit 50-1.

FIG. 46 shows a situation in which the display unit 42 a of the trafficlight 40 a is illuminating display lamps showing that it is possible toturn left or to go straight ahead, while traffic must wait to turnright. The control unit 46 of the traffic light for automobiles 40 aperforms control so as to raise the frame rate of the unit regions ofthe imaging element 100 that correspond to the lane of travel A alongwhich vehicles that are going straight ahead or turning left pass and tothe lane of travel B along which vehicles that are going straight aheadpass to be higher than the frame rate of the unit regions thatcorrespond to the other lane of travel C, and so as to lower theirdecimation ratio. To put this in another manner, the control unit 46performs control so as to set lower the frame rate of the unit regionsthat correspond to the lane of travel C, and so as to set theirdecimation ratio higher.

It should be understood that the concept of lowering of the frame ratealso includes making a setting so that image capture is not performed bythe unit regions in question.

In the control that has been explained above according to the embodimentexplained above of the image capture conditions for the imaging elements100 of the automobile 10 or of the traffic light 40 (either the trafficlight for automobiles 40 a or the traffic light for pedestrians 40 b),it would be acceptable to arrange to change the image capture conditionsalmost simultaneously with changing the timing of changeover of thedisplay unit 42, or alternatively it would also be acceptable to arrangeto change the image capture conditions by providing a fixed timeinterval gap from the timing at which the display is changed over. Or,it would also be acceptable to arrange to change the image captureconditions by, for a fixed time period directly after changeover of thedisplay, taking both the region of attention that was set beforechangeover of the display and also the region of attention that is to beset after changeover of the display as regions of attention.

According to this third embodiment, the following beneficial operationaleffects are obtained.

(1) The image capture device of the automobile 10 (or of the trafficlight 40) comprises the image capture unit 5 (or the image capture unit50) comprising the imaging element 10 that is capable of setting imagecapture conditions for a plurality of regions, and the control unit 19(or the control unit 41) that sets image capture conditions for theplurality of regions on the basis of the driving mode of another vehicle20 in the vicinity. Due to this, it is possible to set image captureconditions for the imaging element 100 that are adapted to the drivingmode of another vehicle 20 in the vicinity.

(2) Since the control unit 19 (or the control unit 41) sets imagecapture conditions that are different for each region that captures animage of another vehicle 20 for which the driving mode is different,accordingly it is possible to set image capture conditions to bedifferent for regions of the imaging element 100 that capture images ofvehicles 20 for which the driving modes are different.

(3) Since the driving modes of other vehicles 20 can be either theautomatic driving mode or the manual driving mode, and since the controlunit 19 (or the control unit 41) sets the image capture conditions to bedifferent for the regions that capture images of a vehicle 20 whosedriving mode is the automatic driving mode and for the regions thatcapture images of a vehicle 20 whose driving mode is the manual drivingmode, accordingly, for the imaging element 100, it is possible to setimage capture conditions that are different between the regions thatcapture images of a vehicle 20 whose driving mode is the automaticdriving mode, and the regions that capture images of a vehicle 20 whosedriving mode is the manual driving mode.

(4) The control unit 19 (or the control unit 41) sets the frame rate ofthe regions that capture images of a vehicle 20 whose driving mode isthe manual driving mode to be higher than the frame rate of the regionsthat capture images of a vehicle 20 whose driving mode is the automaticdriving mode. Since, due to this, the frequency with which images arecaptured of a vehicle 20 whose driving mode is manual is increased to behigher than the frequency with which images are captured of a vehicle 20whose driving mode is automatic, accordingly it is possible to enhancethe level of attention that is given to vehicles 20 whose driving modeis manual. In other words it becomes possible to acquire quicker andmore accurate information in relation to the behavior of vehicles 20whose driving mode is manual, which is behavior that it is not possibleto predict.

(5) The control unit 19 (or the control unit 41) sets the decimationratio of the pixels in the regions that capture images of a vehicle 20whose driving mode is the manual driving mode to be lower than thedecimation ratio of the pixels in the regions that capture images of avehicle 20 whose driving mode is the manual driving mode. Due to this,it is possible to make the amount of information for vehicles 20 whosedriving mode is automatic to be greater than for vehicles 20 whosedriving mode is manual. In other words, it becomes possible to acquiremore accurate information in relation to the behavior of vehicles 20whose driving mode is manual, which is behavior that it is not possibleto predict.

(6) Since it is arranged to provide the control unit 19 (or the controlunit 41) that acquires information related to the method of driving ofother vehicles 20 in the vicinity, accordingly it is possible to setimage capture conditions for the imaging element 100 on the basis of thenewest information that has been acquired, even if, for example, somenew other vehicle 20 in the vicinity has replaced a previous one.

(7) Since it is arranged for the control unit 19 (or the control unit41) to acquire information by communication with another vehicle 20,accordingly it is possible to set the image capture conditions for theimaging element 100 in an appropriate manner on the basis of newinformation that has been acquired by such communication.

(8) Since the control unit 19 (or the control unit 41) acquiresinformation by capturing an image of the other vehicle 20 with the imagecapture unit 5 (or with the image capture unit 50), accordingly, even ina state in which communication is not possible, still it is possible toset the image capture conditions for the imaging element 100 in anappropriate manner on the basis of new information.

(9) The control unit 19 acquires information from the traffic light 40that is different from other vehicles 20. Due to this, even in a statein which communication with other vehicles 20 is not possible, still itis possible to set the image capture conditions for the imaging element100 in an appropriate manner on the basis of new information.

(10) The control unit 19 (or the control unit 41) acquires informationindicating whether the other vehicle 20 is an automatically drivenvehicle or is a manually driven vehicle, and sets image captureconditions that are different for regions that capture an image of anautomatically driven vehicle and for regions that capture an image of amanually driven vehicle. Due to this it is possible to set image captureconditions in an appropriate manner for each unit region on the imagingsurface of the imaging element 100, so as, for example, to enhance thelevel of attention accorded to a manually driven vehicle as compared tothat accorded to an automatically driven vehicle. In particular, byincreasing the frame rate and reducing the pixel decimation ratio for amanually driven vehicle, it is possible to acquire information in aquick and accurate manner about the behavior of a manually drivenvehicle, which is behavior that it is not possible to predict.

(11) Since the automobile 10 is provided with the image capture devicethat provides the benefits (1) through (10) described above, accordinglyit is possible to set the image capture conditions for the image capturedevice in an appropriate manner that is matched to the methods ofdriving of other vehicles 20 in the vicinity of the automobile 10.

(12) The control unit 19 (or the control unit 41) of the automobile 10sets the image capture conditions for the imaging element 100 accordingto actuation of the steering wheel, of the turn signal switch, of theaccelerator, and/or of the brake of the automobile 10. Due to this, itis possible to set image capture conditions in an appropriate manner foreach of the unit regions on the imaging surface of the imaging element100, according to change of the course of the automobile 10, or changeof its speed or the like.

(13) The traffic light 40 includes the image capture unit 50 comprisingthe imaging element 100 that is capable of setting image captureconditions for a plurality of regions independently, and the controlunit 46 that sets the image capture conditions for a plurality ofregions on the basis of information related to the movement of theautomobile 10 and of the other vehicle 20. Due to this, it is possibleto set the image capture conditions for the imaging element 100according to the automobile 10 that is moving at an intersection or thelike, and according to the state of movement of other vehicles 20.

(14) Information related to the movement of the automobile 10 and othervehicles 20 includes signals that permit movement of the automobile 10and the other vehicles 20, and signals that do not permit movement ofthe automobile 10 and the other vehicles 20, and it is arranged for thecontrol unit 46 to set the image capture conditions on the basis ofthese various signals. Due to this, it is possible to set the imagecapture conditions for the imaging element 100 in a manner that isappropriate for when the automobile 10 and the other vehicles 20 aremoving, and also for when the automobile 10 and the other vehicles 20are not moving.

(15) The signals that permit movement of the automobile 10 and the othervehicles 20 include signals that permit movement straight ahead, signalsthat permit turning left, and signals that permit turning right, and itis arranged for the control unit 46 to set the image capture conditionson the basis of each of these types of signal. Due to this, it ispossible to set the image capture conditions for the imaging element 100in an appropriate manner for each of when the automobile 10 and theother vehicles 20 are going straight ahead, are turning left, or areturning right.

(16) Since the control unit 46 changes the image capture conditionsaccording to changing over of the information related to the movement ofthe automobile 10 and the other vehicles 20, accordingly it is possibleto change the image capture conditions for the imaging element 100 in anappropriate manner at the timing at which the signals described abovechange over.

(17) Since the traffic light 40 includes the control unit 46 thatacquires information related to the movement of the automobile 10 andthe other vehicles 20 in the vicinity, accordingly even if, for example,some other vehicle 20 in the vicinity is replaced by another, it isstill possible to set the image capture conditions for the imagingelement 100 on the basis of the newest information that has beenacquired.

(18) Since the control unit 46 acquires information by communicationwith the automobile 10 and other vehicles 20, accordingly it is possibleto set the image capture conditions for the imaging element 100 in anappropriate manner on the basis of new information that has beenobtained by communication.

(19) Since the control unit 46 acquires information by capturing imagesof the automobile 10 and other vehicles 20 with the image capture unit50, accordingly it is possible to set the image capture conditions forthe imaging element 100 in an appropriate manner on the basis of newinformation, even in a situation in which communication is not possible.

(20) As information related to movement, the control unit 46 acquiresinformation that indicates whether the automobile 10 and the othervehicle 20 are automobiles that are being driven in the automaticdriving mode or are automobiles that are being driven in the manualdriving mode, and sets image capture conditions that are different forregions that capture images of automobiles that are being driven in theautomatic driving mode, and for regions that capture images ofautomobiles that are being driven in the manual driving mode. Due tothis it is possible to set image capture conditions for each of the unitregions of the imaging surface of the imaging element 100 in anappropriate manner, so as, for example, to enhance the level ofattention that is accorded to automobiles that are being driven in themanual driving mode, as compared to automobiles that are being driven inthe manual driving mode. In particular, by increasing the frame rate andreducing the pixel decimation ratio for automobiles that are beingdriven in the manual driving mode, it is possible to acquire informationin a quick and moreover accurate manner related to the behavior ofautomobiles that are being driven in the manual driving mode, which isbehavior that cannot be predicted.

(21) The control unit 46 acquires information specifying changes ofcourse of the automobile 10 or of other vehicles 20, and changes theregions for which image capture conditions are set on the basis of suchchanges of course of the automobile 10 or of other vehicles 20. By doingthis, it is possible to change the image capture conditions of theimaging element 100 in an appropriate manner, at the timing of suchchanges of course.

(22) Since the communication unit 41 is provided that performscommunication with other automobiles 10 and with other vehicles 20 thatare different from the automobile 10 and the other vehicle 20,accordingly it is possible to transmit information about other vehicles20 in the neighborhood of the traffic light 40 to the automobile 10.

(23) The automobile 10 and the traffic light 40 measure the speed atwhich an object whose image has been captured is moving, and change theimage capture conditions for the imaging element 100 according to themagnitude of this speed and its direction and orientation. Accordinglyit is possible to change the image capture conditions for each regionupon the imaging surface of the imaging element 100 in an appropriatemanner, according to the state of movement of the object.

(24) The automobile 10 acquires information such as in how many secondsa signal will change or the like, and reflects this information in thestate of driving of the automobile 10. Accordingly it is possible toimplement smooth driving in which changing over of the signal is takeninto consideration in advance.

(25) For a fixed time period directly after the signal changes over, theautomobile 10 or the traffic light 40 sets image capture conditions forthe imaging element 100 so as to include, as regions of attention, boththe region of attention that was set for the signal directly before thechangeover and also the region of attention that must be set to thesignal after changeover. Due to this, it is possible to set the imagecapture conditions for the imaging element 100 in an appropriate mannerfor the transient state during changeover of the signal.

(26) With this image capture system 1 that includes the automobile 10and the traffic light 40, it is possible to implement a more organizedtraffic system on the basis of information acquired by accurate andefficient image capture and on the basis of communication of thatinformation.

It should be understood that while, in this third embodiment, the imagecapture unit 5 and the image capture unit 50 are respectively controlledby the control unit 19 of the automobile 10 and by the control unit 46of the traffic light 40, it would also be acceptable to arrange forparts of the control of the image capture unit 5 and the image captureunit 50 to be performed by control circuits (CPUs dr the like) interiorto the image capture units.

Moreover, it would also be acceptable to arrange for part of theprocessing performed by the control unit 46 of the traffic light 40 tobe performed by the control unit 34 of the signal information generationdevice 30. And the image capture unit 50 of the camera or the like isnot necessarily installed directly to the traffic light 40; it may beinstalled in some other location, according to the situation with thetraffic signals or of traffic at the intersection or the like.

Furthermore while, in this third embodiment, the display unit and theaudio replay unit of the navigation system 13 were used for providingmessages, it would also be acceptable to arrange to utilize a separatedisplay and replay device. Yet further, it would also be acceptable toarrange to utilize a display and replay device that consists of a HUD(Heads Up Display) that projects information upon the front windshieldof the automobile 10, and a speaker that replays audio information.

One or a plurality of the variant embodiments described below may alsobe combined with the third embodiment described above.

Variant Embodiment #1

It would also be acceptable for the control unit 46 of the traffic light40 to acquire, by image captured by the image capture unit 50 or bycommunication via the communication unit 41, information related to theproportion of vehicles upon a road or at an intersection that areautomatically driven vehicles, and to change the image captureconditions according to that proportion. For example, the control unit46 may control the decimation ratio to be greater in time slots in whichthe proportion of automatically driven vehicles is high, as compared totime slots in which the proportion of automatically driven vehicles islow. By doing this, it is possible to implement economy of theconsumption of electrical power and so on, and to perform image capturein a more efficient manner.

Variant Embodiment #2

In addition to the control unit 19 of the automobile 10 or the controlunit 46 of the traffic light 40 identifying whether a vehicle is anautomatically driven vehicle or a manually driven vehicle by imagecapture, it would also be acceptable to arrange to identify vehiclesigns such as novice driver vehicle signs or senior driver vehicle signsor the like. And the image capture conditions may be set differently forunit regions of the imaging element 100 that correspond to novice drivervehicles and for unit regions that correspond to senior driver vehicles.For example, it would be acceptable to arrange for the image captureframe rate for unit regions of the imaging element 100 that correspondto vehicles that are displaying novice driver signs to be set yet higherthan the frame rate for unit regions that correspond to manually drivenvehicles. By doing this, it is possible to set the image captureconditions for each of the unit regions upon the imaging surface of theimaging element 100 in an appropriate mariner, according to each object.

Variant Embodiment #3

While, in the third embodiment, measurement of distances and detectionof surrounding moving objects and/or obstructions were performed byimage capture using the imaging element 100, it would also be acceptableto arrange to use a radar not shown in the figures in paralleltherewith. By doing this, it becomes possible to acquire more reliabletraffic information by making full use of the characteristics of theimaging element 100 and the radar.

While various embodiments and variant embodiments have been explained,the present invention is not to be considered as being limited by thedetails thereof. Modes in which the various structures disclosed inthese embodiments and variant embodiments are employed in combinationare also included in the range of the present invention. And other modesthat are considered to come within the range of the technical concept ofthe present invention are also to be considered as being included withinthe scope of the present invention.

The contents of the disclosures of the following applications, uponwhich priority is claimed, are hereby incorporated herein by reference:

Japanese Patent Application No. 2014-111375 (filed on 29 May 2014).

Japanese Patent Application No. 2014-173833 (filed on 28 Aug. 2014).

Japanese Patent Application No. 2015-005171 (filed on 14 Jan. 2015).

International Publication WO13/164,915.

REFERENCE SIGNS LIST

-   1: vehicle-   2: driving support device-   3: camera-   4: control device-   4 b: storage unit-   5: first travel control unit-   6: second travel control unit-   7: throttle control device-   7 a: accelerator pedal-   8: brake control device-   8 a: brake pedal-   9: steering control device-   10: steering wheel-   11: turn signal switch-   12: vehicle speed sensor-   14: display device-   15: UPS device-   16: shift lever position detection device-   17: microphone-   18: beam changeover switch-   19: rainfall sensor-   31: image capture optical system-   32: image capture unit-   32 a: drive unit-   33: image processing unit-   34: working memory-   35: control unit-   35 a: range finding calculation unit-   36: recording unit-   60: focus detection pixel line-   71, 73, 75, 77, 79, 79A: first imaging regions-   71 a: third imaging region-   72, 74, 76, 78, 80: second imaging regions-   72 a: fourth imaging region-   73 a, 73 c: fifth imaging regions-   74 a, 74 c: sixth imaging regions-   81: imaging region-   82, 82A, 82B: regions of attention-   83: inactive region-   87: seventh imaging region-   100: imaging element-   113: image capture chip-   1: image capture system-   5, 50 (50-1, 50-2): image capture units-   10: automobile-   19, 46: control units-   20: other vehicle-   30: traffic signal generation device-   40: traffic light-   40 a: traffic light for automobiles-   40 a: traffic light for pedestrians-   42: display unit-   70: imaging region-   71, 71A, 71B: regions of attention-   72, 72A, 72B, 73A, 7313, 76, 77: vehicles-   74A, 74B: secondary regions of attention-   75: special region of attention-   90: pedestrian

1-51. (canceled)
 52. An image capture device that is mounted to avehicle, comprising: an image capture unit that has a plurality ofimaging regions, an imaging condition for each imaging region beingchangeable; and a setting unit that sets an imaging region used forcontrol of the vehicle and an image capture condition for the imagingregion, based upon at least one of a state of the vehicle and a state ofa photographic subject that is other than the vehicle and is captured bythe image capture unit.
 53. The image capture device according to claim52, wherein: the state of the vehicle includes a traveling state of thevehicle.
 54. The image capture device according to claim 53, wherein:the traveling state of the vehicle includes a direction of travel of thevehicle.
 55. The image capture device according to claim 54, wherein:the setting unit sets the imaging region used for control and the imagecapture condition for the imaging region used for control, based upon arotational angle of a steering wheel as the direction of travel of thevehicle, the rotational angle being an amount of an angle by which thesteering wheel is turned from a position for the vehicle to go straight.56. The image capture device according to claim 55, wherein: the settingunit sets a position of the imaging region used for control, based uponthe rotational angle of the steering wheel of the vehicle.
 57. The imagecapture device according to claim 56, further comprises: an input unitto which information concerning a pedestrian in a vicinity of thevehicle is input, wherein: the setting unit sets an image capturecondition for an imaging region in which an image of the pedestrian iscaptured as information on a distance between the vehicle and thepedestrian changes.
 58. The image capture device according to claim 57,wherein: the setting unit changes the image capture condition for theimaging region in which the image of the pedestrian is captured as theinformation on the distance between the vehicle and the pedestrianchanges.
 59. The image capture device according to claim 55, wherein:the image capture condition for the imaging region includes a framerate; and the setting unit sets the frame rate for the imaging regionused for control, based upon the rotational angle of the steering wheelof the vehicle.
 60. The image capture device according to claim 59,wherein: the setting unit increases the frame rate for the imagingregion used for control, based upon an amount of the rotational angle ofthe steering wheel of the vehicle.
 61. The image capture deviceaccording to claim 53, wherein: the traveling state of the vehicleincludes speed of the vehicle.
 62. The image capture device according toclaim 61, wherein: the setting unit sets a position of the imagingregion used for control based upon the speed of the vehicle.
 63. Theimage capture device according to claim 61, wherein: the image capturecondition for the imaging region includes a frame rate; and the settingunit sets the frame rate for the imaging region based upon the speed ofthe vehicle.
 64. The image capture device according to claim 63,wherein: the setting unit increases the frame rate for the imagingregion based upon the speed of the vehicle.
 65. The image capture deviceaccording to claim 64, wherein: the setting unit increases the framerate for the imaging region with increase in the speed of the vehicle.66. The image capture device according to claim 64, wherein: the settingunit increases the frame rate for the imaging region as the vehicledecelerates.
 67. The image capture device according to claim 52,wherein: the state of the photographic subject that is other than thevehicle and is captured by the image capture unit includes a distancebetween the vehicle and the photographic subject.
 68. The image capturedevice according to claim 67, wherein: the distance between the vehicleand the photographic subject is a distance between the vehicle andanother vehicle other than the vehicle.
 69. The image capture deviceaccording to claim 68, wherein: the setting unit sets a decimation ratiofor signals to be read from the imaging region used for control as theimage capture condition for the imaging region used for control, anddecreases the decimation ratio based on the distance between the vehicleand the other vehicle.
 70. The image capture device according to claim69, wherein: the setting unit increases a size of the imaging regionused for control as the distance between the vehicle and the othervehicle becomes shorter.
 71. The image capture device according to claim52, wherein: the state of the photographic subject that is other thanthe vehicle and is captured includes a luminance of the photographicsubject which is captured by the image captured unit.
 72. The imagecapture device according to claim 71, wherein: as the image capturecondition for the imaging region used for control, the setting unit setsa frame rate for the imaging region used for control to be higher than aframe rate for an imaging region other than the imaging region used forcontrol.
 73. The image capture device according to claim 72, wherein: asthe image capture condition for the imaging region used for control, thesetting unit increases an exposure for a part of the photographicsubject in the imaging region used for control if the luminance of thepart of the photographic subject is lower than that of another part ofthe photographic subject.
 74. The image capture device according toclaim 73, wherein: as the image capture condition for the imaging regionused for control, if the photographic subject contains an entry to atunnel, the setting unit increases an exposure for a portioncorresponding to the entry to the tunnel in the imaging region used forcontrol.
 75. The image capture device according to claim 74, wherein:the setting unit increases the exposure for the portion corresponding tothe entry to the tunnel by decreasing a frame rate for the portioncorresponding to the entry to the tunnel.
 76. An image capture devicethat is mounted to a vehicle, comprising: an image capture unit that hasa plurality of imaging regions, an imaging condition for each imagingregion being changeable; and a setting unit that sets an imaging regionused for control of the vehicle, based upon information about anexterior of the vehicle.
 77. The image capture device according to claim76, wherein: the information about the exterior of the vehicle includesinformation concerning change of a vehicle lane for the vehicle.
 78. Theimage capture device according to claim 77, wherein: the informationconcerning change of the vehicle lane for the vehicle is obtained from anavigation device.
 79. The image capture device according to claim 78,wherein: the setting unit sets an image capture condition for theimaging region used for control, based on the information concerningchange of the vehicle lane for the vehicle.
 80. The image capture deviceaccording to claim 52, wherein: the setting unit sets an imaging regionthrough which the vehicle passes as the imaging region used for controlof the vehicle.
 81. An image creating device, comprising: an imagecreation unit that creates an image including a first region with afirst resolution and a second region with a second resolution; a sightline detection unit that detects a sight line; and a control unit thatcontrols at least one of the first resolution at the first region andthe second resolution at the second region based on a detection resultby the sight line detection unit.
 82. The image creating deviceaccording to claim 81, wherein: the control unit sets the secondresolution at the second region to be higher than the first resolutionat the first region, based on the detection result by the sight linedetection unit.
 83. The image creating device according to claim 81,further comprises: a position change unit that changes a position of thesecond region within the image, based on the sight line.
 84. An imagecreating method, comprising: creating an image including a first regionwith a first resolution and a second region with a second resolution;detecting a sight line; and controlling at least one of the firstresolution at the first region and the second resolution at the secondregion based on a detection result of the sight line.
 85. The imagecreating method according to claim 84, wherein: the second resolution atthe second region is set to be higher than the first resolution at thefirst region, based on the detection result of the sight line.
 86. Theimage creating method according to claim 84, wherein: a position of thesecond region within the image is changed based on the sight line. 87.An image creating device, comprising: an image creation unit thatcreates an image including a first region with a first resolution and asecond region with a second resolution that is higher than the firstresolution; a sight line detection unit that detects a sight line; and aposition change unit that changes a position of the second region withinthe image, based on the sight line.
 88. The image creating deviceaccording to claim 87, wherein: the position change unit changes aposition of the first region within the image, based on the sight line.89. An image creating method, comprising: creating an image including afirst region with a first resolution and a second region with a secondresolution that is higher than the first resolution; detecting a sightline; and changing a position of the second region within the image,based on the sight line.
 90. The image creating method according toclaim 89, wherein: a position of the first region within the image ischanged based on the sight line.
 91. A vehicle to which the imagecapture device according to claim 52 is mounted.
 92. An image capturedevice that is mounted to a vehicle, comprising: an image capture unitthat has a plurality of imaging regions, an imaging condition for eachimaging region being changeable; and a setting unit that sets an imagingregion used for supporting a driver of the vehicle and an image capturecondition for the imaging region, based upon at least one of a state ofthe vehicle and a state of a photographic subject that is other than thevehicle and is captured by the image capture unit.
 93. An image capturedevice that is mounted to a vehicle, comprising: an image capture unitthat has a first imaging region and a second imaging region that isdifferent from the first imaging region, an image capture condition forthe first imaging region being changeable, positions of the firstimaging region and the second imaging region being changeable; and asetting unit that sets the first imaging region used for control of thevehicle and the second imaging region and the image capture conditionfor the first imaging region, based upon at least one of a state of thevehicle and a state of a photographic subject that is other than thevehicle and is captured by the image capture unit.